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AGENT BASED IMSTRUCTION 
SYSTEM AKD METHOD 



1. 



FIELD OF THE IWVEKTION 



This invention relates to a system and method for 
interactive, adaptive, and individualized computer-assisted 
instruction of students, preferably implemented on network 
connected computers. More particularly the system and method 
includes for each student an agent adapted to that student 
which monitors its student's instructional behavior, responds 
to teacher direction, and controls the instructional 
progress, and guides its student, all of which constitute one 
aspect of a virtual tutor. Preferably, the viewable on- 
screen aspect of the agent includes customizable multimedia 
presentation personae, which constitute a further aspect of a 
virtual tutor. 



The application of computers in education has been 
limited by several problems, including a failure to provide 
systems that adapt or individualize to each student, a 
failure to integrate systems effectively into the existing 
classroom in elementary and secondary schools, and a failure 
to exploit technological developments. Although during the 
last three decades, a number of interactive educational 
techniques have been implemented on computers, all these 
systems lack the ability to recognize and to adapt to each 
student's individual characteristics. The comjnon motivation 
for interactive educational techniques was the recognition 
that individual student interaction fosters learning to a 
greater degree than mere passive exposure to a fixed pace 
presentation (Kulik et al*, 1986, Effectiveness of computer- 
based adult learning: a meta-analysis. Journal of educatlo.ial 
computing resBarch 2:235-252; Kulik et al. , 1983, Effects of 
computer-based teaching on secondary school students. Journal 
of educational psychology 25:19-26). Existing, interactive 
educational techniques have many variants: programmed 
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instruction, mastery learning, audio-tutorials, direct 
instruction, personalized system of instruction, precision 
teaching, fluency learning and others (Engleman et al., 1982, 
Theory of Instruction: Principles and Applications , 
5 Irvington, New York; Keller, 1968, "Goodbye, teacher..,". 
Journal of Applied Behavior Analysis 1:79-89; Lindsley, 1990, 
Precision teaching: By teachers for children. Teaching 
Exceptional Children 21:353-359; West et al. , 1992, Designs 
for Excellence in Education: the Legacy of B.F. Skinner, 

10 Sorris West, Inc., Longmont, Colorado, pp, 147-160). Several 
systems have attempted to harness the interactivity of the 
computer for these interactive educational techniques. in 
early work, for example, text-based programmed instruction 
was converted to computer format and implemented on time- 

15 shared systems. This early development was extended with 
more sophisticated computer-assisted instruction ("CAI") , 
also known as compute based training ("CBT**) . 

In CAI , for example, the computer acts as a teaching 
machine. A program presents instructional displays, accepts 

20 student responses, edits and judges those responses, branches 
on the basis of student responses, gives feedback to the 
student, and records and stores the student's progress. 
Examples of CAI systems include those of Carbonell, 1970, AI 
in CAI, an artificial intelligence approach to computer- 

25 assisted instruction, IEEE Transactions on Man-machine 
Systems, 11:190-202; Osin, 1984, CAI on a national scale, 
Proc . 4 th Jerusalem Conf. on Information Technology, pp 418- 
424; Seidel 1971; Koffman et al., 1975, Artificial 
intelligence and artificial programming in CAI, Artificial 

30 Jnteiligence 6:215-234. Effective CAI instructional 

materials for a limited number of specific topics have been 
developed, as have special "authoring languages," which 
assist instructional developers on the tasks of designing 
instructional materials* U.S, Patent No. 5,310,349 is 

35 exemplary of such CAI systems. 

However, existing CAI systems do not adapt to their 
students. These systems merely sequence students through 
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educational materials, based only on student performance 
during a current lesson and using only parameters such as 
recent responses and pre-requisite patterns. These systems 
do not gather or use information on more comprehensive 
5 student characteristics, such as past student performance, 
student performance on other courses, student learning 
styles, and student interests, 

A greater deficiency is that existing CAI systems do not 
recognize characteristics of their individual students. They 

10 cannot be individualized or made responsive to their students 
styles. Thereby, these system ignore those roles of a human 
tutor that can be of unparalleled significance in the 
education of an individual. The human tutor assists in 
scheduling and prioritizing and in maintaining interest 

15 through proper reinforcement and knowledge of student 
abilities and preferences. A human tutor observes and 
addresses patterns of errors and maintains a consistent 
manner of interaction across a broad range of subject matters 
and activities. Moreover, a human tutor effectively 

20 integrates the cognitive, personal and social aspects of the 
instructional situation. In other words the human tutor 
provides a level of individualization based on student styles 
and on requirements of the instructional task. Furthermore, 
the human tutor provides an equally effective interaction 

25 with the teacher by accepting individualized instructions, 
collecting data and providing detailed reports. By failing 
to address these higher order services and roles of an 
effective human tutor, existing CAI systems fail to fully 
engage their students and thus fail instruct as well as 

30 possible. 

Additionally, a further problem of computer assisted 
instruction, particularly in primary and secondary school 
settings is poor integration into the rest of the school 
curriculum and often poor quality of the educational 
35 materials. In application to elementary and secondary 
schools, two main patterns of instructional computer use 
prevail, which illustrate the compromises prevalent today. 
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In one pattern, the integrated learning system ("ILS") is a 
dedicated installation that is used in schools to teach basic 
strands of reading, mathematics and related topics, spelling, 
writing, and other language arts, from grades one to six, or 
5 perhaps to eight or nine (EPIE, 1990, Integrated 

Instructional Systems: an Evaluation, Educational Products 
Information Exchange, Hampton Bays, N.Y.)- The paradox with 
this pattern, regardless of the quality of the instruction 
offered by these systems, the work of students in ILSs bears 

10 little relation to what goes on in the classrooms in that 
very same topic • The fundamental reason is that the teacher 
cannot influence or respond meaningfully to variations in 
student progress or to relations between the CAI curriculum 
and the classroom text, materials and activities. This is 

15 the case even where the ILS installation produces reports in 
some detail as to each student's progress and standing. 

The second pattern of computer use in schools is that of 
stand-alone short units on specific topics (TESS, 1996, 
Datajbase of Educational Software Systems, Educational 

2 0 Products Information Exchange, Hampton Bays, N.Y.). These 
can be hypercard programs, simulations, or games, and are 
usually separate from the basic classroom curriculum. Though 
occasionally of excellent quality, the paradox here is that 
these products are usually chosen for enhancement, possible 

25 optional, and do not account for a major component of school 
related instruction. No records are kept or returned 
regarding student performance. These programs have not made 
a major contribution to school instruction. 

Finally, computer assisted instruction systems have 

30 ignored or under utilized such important developments in 
computer technology over the past four years as agent-based 
system, client-server systems, and networking systems. 
Though now an active field with a wide spectrum of activities 
from research to commercial applications, application of 

35 agent-based systems in educational, instructional, and 

homework tasks has not been explored. Software systems for 
intelligent agents have successfully applied in travel 
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arrangements, email manageinenr, meeting scheduling, stock 
portfolio management, and gathering information from the 
Internet (Maes, 1994, Agents that reduce work and information 
overload. Communications of the ACM 32:30-40), In all these 
5 applications, software agents perform tasks on the user's 
behalf, receiving only general instructions from their user 
but then executing detailed tasks with considerable 
independence and initiative- In client-server systems, these 
agents can operate in the client, the server, or both. 

10 Recently, adaptive and personalized agent based systems 

have begun to be developed. Systems with adaptive agents, 
agents which learn from experience, has made gains with new 
techniques continually identified. Adaptive agents have 
permitted new commercially viable adaptive systems 

15 implemented across networks. In these systems, an agent is a 
"go-between," mediating relations in a manner whose function 
is understood with details being left to the agent itself. 
The agent acts as a "stand-in" for its user, who is thus 
freed from direct manipulation of the network. In 

20 instructional applications, there is an unmet need for an 
agent who serves two users: the school system and the 
individual student. This is the well-known role of the 
teaching assistant/ tutor . Maes, 1994, and others have 
extended the metaphor of agent to that of personal assistant, 

25 an agent who learns some important characteristics of its 
user, and adapts its behavior accordingly. Agents can learn 
by a mixture of methods: observation, receiving feedback 
from its user, receiving instructions from the user, and 
consulting other agents concerning "similar problems." To 

30 combine the important properties of competence, trust, and 
intimacy that a personal assistant should have, an agent 
should be in touch with relevant data, represent important 
facts in a reliable manner, and engage with its user in a 
personal and fundamentally sympathetic — at times playful — 

35 manner. Approaches to the creation of agents with personal 
characteristics have begun to be explored. In this work, 
relevant techniques are found in the tradition of film 
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animators who, through the portrayal of emotions, gave their 
characters the illusion of life. 

Moreover, computer assisted instructional systems have 
only haphazardly exploited the potential of client-server 
5 systems and networking technologies. Client-server 

architectures have emerged as the principal architecture of 
distributed computer systems. Client systems, running under 
sophisticated windowing operating systems, can support 
advanced object based software applications, including high 

10 speed graphics, animation and audio output. Servers can 
store gigabytes of data and programs at central or 
distributed locations at quite reasonable cost. Object 
oriented database systems have been developed to store 
structured data on servers. 

15 Client systems, in a striking change from only several 

years ago, now virtually all have multimedia capabilities, 
including high quality graphics, sound, and at least limited 
video playback capability. Text-to-speech software is 
presently available for use with these systems, and speech 

2 0 recognition software is on brink of widespread commercial 
acceptability on low cost platforms. New authoring tools 
support graphical methods for generation of multimedia 
presentations and computer based instructional materials 
having corresponding sequencing logic. 

25 Clients and servers can be linked remotely with 

increasing convenience and decreasing cost. The Internet has 
emerged as a means of providing an inexpensive means of 
connecting computers to provide effective communications and 
access to information and other resources such as software. 

30 Further Internet developments that made the Internet truly 
universal include the HTML and the HTTP protocols, which 
provide platfoirm independent access to hyperlinked multimedia 
information, and the Java"* programming language, which 
provides platform independent software for Internet 

35 applications programming. Subsets of the Internet — 

intranets — have become an increasingly important means for 
disseminating information and enabling -communication within 
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constrained domains, such as a single school system or 
corporate enterprise . 

Existing CAI systems have not addressed these functional 
deficiencies nor have they exploited the possibilities of 
5 existing technologies. 

Citation of references hereinabove shall not be 
construed as an admission that such a reference is prior art 
to the present invention. 

3, SUMKARY OP THE IHVBNTION 

The Agent Based Instruction ("ABI") system of this 
invention is a system and method for interactive, adaptive, 
and individualized computer-assisted instruction and 
homework, preferably implemented on network connected 

15 computers, that overcomes these problems by providing the 
following objects in preferred and alternative embodiments. 
This invention provides a more effective system responsive to 
the needs of several parties interested in education. 

An important object of this invention is to provide the 

20 student with a virtual tutor, by having agent software 

("agent") adapted to each student that offers a high quality 
of individualized student interaction and that manages or 
controls instruction in a manner approximating a real tutor. 
The agent exercises management or control over the computer- 

25 assisted instruction materials and provides information and 
help to the student, both synchronously and asynchronously to 
particular instructional materials. Agent behaviors are 
sensitive to both the educational context and to the history 
of student behavior. 

30 In a preferred embodiment of this invention the agent 

integrates data from several sources. From computer-assisted 
instructional materials, it accepts data on the methods of 
instruction adopted by particular materials and on student 
performance in the instruction. From the student, it accepts 

3 5 direct interactions as well as using the history of previous 
student performance stored in a student data object. From 
the teacher, it accepts data on customization and student 
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assignments. From the school, it accepts data on assigned 
courses, data on analysis of student body performance, and 
educational standards and criteria. In a preferred 
embodiment, these inputs allow individualization of agent 
5 interaction. Alternative embodiments are responsive to 
additional data types and sources. 

In a preferred embodiment of this invention, diverse 
agent behaviors are handled uniformly by a single means. The 
diverse behaviors include encouragement and feedback, 

10 providing meta-cognitive help on ongoing instruction, 

managing or controlling and individualizing computer based 
instruction to the student's learning modes, and assistance 
with assignment management. These diverse behaviors are 
selected from a set of potentially appropriate candidate 

15 behaviors. This set of candidate behaviors is ordered and 
the highest ranked behaviors are chosen. 

In a preferred embodiment of this invention the diverse 
agent behaviors adapt to the student based on a variety of 
information about the student. The agent modifies its 

2 0 behavior on the basis of a growing history of interactions 
vith the student over time, as this history of student 
performance is stored in the student data object. The agent 
can also modify its behavior on the basis of teacher and 
school system supplied information. 

2 5 Further, in this preferred embodiment, the student data 

object resides on portable media in the physical control of 
the student. This is advantageous because the student data 
object accumulates data personal to the student, and its 
privacy can be ensured by such student control. 

3 0 Alternatively, the student data object can be stored on 

central server computers in secure databases when this is 
advantageous to ensure student data privacy. 

Another important object of this invention is that the 
agent presents itself on-screen to the student with 
35 integrated, and optionally, animated multimedia persona, or 
preferably a plurality of persona (hereinafter called 
"personae") . The on-screen agent can appear as living 
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entities, which in grade school can be comfortable "Study 
Buddies'"" and in adult training can be appear as an objective 
"Concept Coach". The on-screen agent instructs, motivates, 
engages and guides its student • 
5 In a preferred embodiment, the on-screen agent can be 

dramatized by a single character or by a cast of interacting 
characters. The interaction between these actors can be 
individualized to reflect the pedagogical response of the 
agent- To maintain student interest in the agent response, 
10 story lines continuing across materials or session can be 
used. 

In a preferred embodiment of this invention the voices, 
gestures and motions of the personae are derived from the 
chosen behaviors, student personae preferences, and the 

15 history of recent behavior by selection from tables 

containing a rich variety of alternative sound and visual 
display objects. All elements of the on-screen agent display 
are then synthesized in an integrated display script calling 
for graphics, animation, video, or sound as appropriate. 

20 These scripts are then bundled into applets, run-time program 
fragments that represent a complete element of performance. 
This display is highly configurable by the student, the 
teacher, or the system administrator. 

It is an advantage of this invention that elements of 

25 the display objects can be created by artists, animators, 
singers, and so forth, as data snips. Pluralities of data 
snips can be stored in libraries of dynamic clip art and then 
installed in an implementation of this invention. In this 
manner the on-screen agent personae have an appropriately 

30 contemporary, realistic, and engaging manner. Data snips 
are, in general, short clips of sound, voice, graphics, 
animation or video, or combinations of these used to 
construct the on-screen agent. A data snip can also be a 
complete pre-f ormatted animated sequence, perhaps in the 

35 format of a talking animated daily cartoon strip. 

Another important object of this invention is that the 
interactive, adaptive, and self -paced computer-assisted 
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instruction and homework provided by this invention is 
available to geographically dispersed students and from 
geographically dispersed schools or education providers. 
Accordingly, the system of this invention is adapted to both 
5 the student data object stored on portable media and to the 
access of other important data across a variety of networks* 
For example, an implementation of this invention as a 
"Homework Network™" can make computer assisted homework 
available to students of all levels at home. In addition to 

10 computers located at residences of students, the student can 
also access homework materials at computers located in youth 
centers, libraries, schools and other locations. A student 
only has to access a networked computer and "plug in" the 
portable student data object to receive instruction, 

15 Alternatively, the student can access the student data object 
across such a network. 

In a preferred embodiment, the network on which this 
invention is implemented as an intranet configured of 
appropriate links and utilizing the known TCP/IP protocol 

2 0 suite, and as appropriate, ATM technologies, including World 
Wide Web, associated browsers, and mail format extensions. 
Implementation over the public Internet is equally preferred 
in cases where extensive connectivity is needed. 

A further important object of this invention is to 

25 utilize augmented computer-assisted instruction materials 

which present to students a variety of interactive, adaptive, 
and self-paced computer-assisted instruction and homework 
materials in a manner which infoirms the agent of a student's 
progress and performance and which permits the agent to 

30 manage or control the materials to the student's pedagogic 
characteristics. Thereby, the ABI system can effectively 
guide and engage students in their educational tasks. 

In a preferred eubodiment , these instructional and 
homework materials are composed of materials data presented 

35 by a materials engine. The materials data includes display 
objects containing the substance of the instruction, logic to 
sequence the display according to student input, and 
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notations. Notations are augmented definitions that serve to 
pass information to the agent concerning the materials and 
the student. For example, notations classify key sections of 
materials which are educationally significant student 
5 actions. Preferably, authoring tools assist in developing 
these augmented instructional materials. Materials tasks and 
sequences are created and entered by instructional designers 
and subject experts. Notations are usually entered by 
instructional designers and can be customized by teachers. 

10 In a preferred embodiment, the information passed in the 

notations is standardized according to an instruction 
materials interface standard. This standard establishes a 
uniform way the materials independent data relating to 
student performance are to be provided to the agent and a 

15 uniform way for the agent to guide the student in a materials 
independent manner. 

A further important object of this invention is to 
provide to the student a range of tools which are integrated 
with the agent in a manner similar to the instructional 

20 materials. These tools include general tools helpful to 
assigned instructional tasks, and special tools for group- 
work and communication and for student scheduling. 

In a preferred embodiment, the general tools include at 
least a calculator, an encyclopedia, a dictionary, a 

25 thesaurus, each appropriate to the several levels of 

students, which can access an ABI implementation. In a' 
preferred embodiment, the group work and communication 
materials allow, when permitted, message exchange, student 
linking into groups for joint work, and student linking into 

3 0 groups for structured work such as contests. In a preferred 
embodiment the student scheduling tool records assigned 
student activities and their priorities. In an embodiment, 
this tool can be consulted by rhe student to view schedules. 
It can be consulted by the system to prescriptively schedule 

35 required activities, to permit student choice, or to permit a 
mixed scheduling initiative. . Finally, it can be consulted by 
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the agent to offer scheduling advice to the student. 
Typically, student assignments are set by a teacher. 

An object of this invention is reporting of student 
performance to students, teachers, parents, administration, 
5 or to other appropriate individuals in a business enterprise 
or other commercial versions. These reports include the 
unique data on the student's pedagogic performance 
accumulated and analyzed by the agent, as well as all the 
usual and expected performance data on specific materials 

10 available in existing computer-assisted instruction systems. 
In a preferred embodiment, data for such reports, called 
herein school-student data, is forwarded by the agent 
pursuant to school policies to school server computers from 
the students' local computer. On the server computers, it is 

15 stored in server databases against which are run such 
reports. Alternatively, this data can be forwarded to 
existing student information systems present at the school. 
In a further alternative embodiment wherein the student data 
objects are centrally stored, this data can be derived from 

20 these objects, which can be stored in an object oriented 
database system. It is an advantage of this invention in a 
school context that parents can have access to current data 
on their children, and thereby play a more informed role in 
their children's education. 

25 Another object of the invention is to utilize current 

technologies for student interaction. When available, this 
invention is adaptable to Network Computers ("NC") . NCs are 
low cost computers specifically designed to access intranets 
or the public Internet. In a current preferred embodiment 

30 and implementation, this invention is adaptable to multimedia 
PCs for some students, and to such special interaction 
technologies as can be advantageous to special students or 
students with special needs. Typical interactive devices 
include keyboards, mice or other pointing devices, voice 

3 5 recognition, joy-sticks, touch activated devices, light-pens, 
and so forth. Other devices, such as virtual reality 
devices, can be added as they become commercialized. 
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It is clear to those of skill in the art that by 
providing interactive, adaptive, and self -paced computer- 
assisted instruction and homework delivered over widely 
available computer networks this invention has immediate 
5 application in public, private, and commercial school 

environment of all levels. Educational research shows that 
instruction and homework of these characteristics improves 
students' educational outcomes* Further, in school contexts 
this invention advantageously provides* immediate access to 
10 student performance and pedagogic characteristics to all 
interested parties, including parents. 

BRIEF DEflCRIPTIOHS OP THE DRAWINGS 

These and other objects, features, and advantages of the 
15 invention will become apparent to those of skill in the art 
in view of the accompanying drawings, detailed description, 
and appended claims, where: 

Fig. 1 illustrates in overview fashion the principal 
functional components of and parties in the ABI system; 
20 Figs. 2A and 2B illustrate in overview fashion an 

implementation of the functional components of Fig. l; 

Fig. 3 illustrates an exemplary student display screen 
of the implementation of Fig. 2; 

Fig. 4 illustrates in more detail exemplary screen 
25 interaction between the on-screen agent and the instructional 
materials of the implementation of Fig. 2; 

Fig. 5 illustrates an exemplary interaction of a student 
with the ABI system implementation of Fig. 2; 

Fig. 6 illustrates in more detail the software 
30 components and hierarchy in the implementation of Fig. 2; 

Fig. 7 illustrates exemplary message flow through the 
implementation of Fig. 2; 

Fig. 8 illustrates agent action processing of Fig. 7 in 
more detail; 

35 Fig. 9 illustrates agent behavior processing of Fig. 7 

in more detail; 
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Figs. lOA and lOB illustrate the structure of student 
data object of Fig. 7 in more detail; 

Fig. 11 illustrates exemplary processing of the student 
data object of Fig. 7; 

5 Fig. 12 illustrates an exemplary sequence of student 

meta-requests and agent meta-responses . 

5* PgTAILBD DEaCRIP TIOM OP THg IMVgKTTnw 

Sec. 5.1 presents a general overview of the Agent Based 
10 instruction system. See. 5.2 describes the preferred 

hardware and operating software configurations. See. 5.3 
describes details of the instructional interface between the 
ABI system and its users. Sec. 5.4 describes in a general 
fashion the software structure of the ABI system with 
15 subsequent sections describing each component in a more 
detailed fashion. Sec. 5.5 describes the instructional 
materials and the tools in a more detailed fashion and Sec. 
5.6 describes the agent in a more detailed fashion. Sec 5.6 
includes detailed description of the preferred interface 
2 0 between the agent and the materials in the ABI system. 

5 • 1 • ABI Svate« Overviaif 

In the following, the systems and methods of this 
invention are described in the context of a school system, 

25 with examples drawn primarily from elementary education. 

This invention is not so limited. it will be apparent to one 
of skill in the relevant arts that this invention can be 
applied at all levels of public and private education, from 
pre-school through university, and to all forms of commercial 

30 or corporate education. In all these contexts, this 

invention has particular utility in making education and 
training available at school, at the office, at home, at 
schools with geographically dispersed students and to 
students at geographically dispersed schools, and at other 

35 types of locations. Further, it will be apparent that this 
invention can be applied in contexts other than education 
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where monitored interactivity and individualization are to be 
provided, as in child care or weight loss. 

The following glossary contains several terms frequently 
used in the Detailed Description of the invention. It is 
5 presented here as an aid in order to initially introduce 
important terms that are more fully described and explained 
in the following sections. 

Agent: agent software together with the data it references 
10 executing in an ABI system. 

Agent Based Instruction ("ABI**) System: the novel 

instructional system of this invention, preferably 
comprising an agent for responsive, adaptive and 
15 individualized instruction of students and a network for 

distribution of instruction, which projects the personae 
of the virtual tutor. 



Agent Software: software modules that generate responsive, 
2 0 adaptive, and individualized behavior in the ABI system, 

preferably implemented according to methods from 
artificial intelligence. 

Applet: an executable program fragment advantageously 
25 downloaded to a client across the network, in the ABI 

system applets are particularly used to represent a 
complete element of on-screen agent actions, or 
performance, (e.g., a character scratching its head and 
saying an utterance) and can reference various data 
30 snips of animation, sound, and scripting information. 

Authoring Tools: programs used by instructional designers to 
develop materials data, such development includes 
inserting notations . 

35 

Cast: a plurality of persona ("personae") representing the 
on-screen agent. 
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Character: an individual persona in the cast of the on- 
screen agent . 

Concept Coach: a possible alternative name for a persona in 
5 the cast of an on-screen agent that is suitable for high 

school and adult students. 

Data Snip: an elementary piece of sound, voice, animation, 
video, or a graphic; data snips can be combined, 
10 preferably by an applet, to represent a complete element 

of on-screen agent action. 

Electronic Learning Friend ("ELF") : a generic name for any 
persona in the cast of an on-screen agent; also a 
15 possible alternative name for a persona suitable for 

middle school students. 

Instructional Materials: the components of a course of 

instruction, such components are selected according to 
20 the course and can include prerequisite tests, pre- 

tests, lessons, and unit tests. 

Materials Data: the content of instructional materials. 

25 Materials Engine: software modules that reference 

instructional materials data and tools data to present 
the instruction and the tools to the student. 

Meta-request: a student request directly to the on-screen 
30 agent, an exemplary request is 'asking for a hint.' 

Meta-response: all responses to a student produced by the 
agent software, as distinguished from presentations by 
instructional materials, tools and communications. 

35 

Network: the hardware and software connecting student client 
computers to school servers in an ABI system; the 
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network connections can comprise fiber optic links, 
wired links, or terrestrial or satellite wireless links. 

Notations: interface information inserted into materials 
5 data that causes the materials engine to send and 

receive messages from the agent software; preferably 
notations create standardized interface messages between 
the agent and the materials. 

10 On-screen Agent: presentation by the agent software on the 
student's display using such media as sound, voice, 
graphics, animation, video, or other multimedia 
modality; this presentation preferably displays one or 
more life-like personae. 

15 

Persona: a character in the cast of an on-screen agent. 
Personae: the collective plural of persona. 

20 School-student Data: data about each student's educational 
progress collected centrally; this data is forwarded by 
the student's agent pursuant to school data collection 
and privacy policies; depending on these policies, such 
data can range from none to essentially the entire 

25 content of the student data object. 

Student Data object: data about each student which the agent 
software references in order to provide responsive, 
adaptive, and individualized instruction to that 
3 0 Student; this data is updated during course of each 

lesson and is advantageously stored as one object, or 
alternatively a few linked objects, on portable media or 
in the ABI system. 



35 Study Buddies™: a possible alternative name for personae in 
the cast of an on-screen agent that is suitable for 
elementary school students. 
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Tools Data: the content of tools supporting particular 

instructional materials; tools can include a dictionary 
a calculator, or an encyclopedia; and so forth, and 
tools data are the content of the dictionary, the 
5 calculator, or the encyclopedia. 



Utterance : 



a text or voiced response by on-screen agent. 



Virtual Tutor: the ABI system components acting together to 
10 emulate a human tutor; from an individual student's 

perspective, the Study Buddies'*, ELF, or Concept Coach 
appears as his or her personal tutor. 

The following sections of the Detailed Description elaborate 
15 each of these terms and describe their connection and 
interaction so that the ABI system presents responsive, 
adaptive, and individualized instruction and assumes the form 
of a virtual tutor. 

5.1.1. yunctional Coaponents 

In view of these objects and advantages. Fig. i 
illustrates the principal actors and the principal functional 
components in an ABI System. These include, generally, 
materials engine 102, agent software 108, and student data 

25 object 109, all of which interact with student lOi and with 
teachers and administrators 106 via a computer network 
described below in conjunction with Fig. 2 to create a 
virtual tutor of student lOl. Student lOl is typically one 
of many students enrolled in a school or similar institution. 

30 Central to the ABI System is the virtual tutor individualized 
to each student, which formed by the functioning of agent 
software 108 with student data object 109, which stores 
characteristics of student lOl and assignments and standards 
set by teachers and administrators 106. Other actors not 

35 shown in Fig. 1 can be relevant in particular applications, 
for example, parents in the case of primary and secondary 
education. 
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Materials engine 102 presents educational content such 
as instructional units, homework assignments, and testing to 
student 101. The educational content includes instructional 
inaterials data 114, communications materials data 104, and 
5 tools data 115. instructional materials data 114 include 
computer based instructional materials similar to those known 
in the art but, importantly, augmented with notations for use 
in this invention. The materials also include various tools 
115 appropriate to particular instructional materials, such 

10 as a calculator for mathematics, a dictionary for writing, 
access to on-line reference sources for research, and so 
forth. Further, materials can also include communication 
materials data 104, which define and provide communication 
with other students 105 for instructional purposes. Such 

15 purposes can provide, for example, the tutoring of one 
student by another more advanced student, joint work of 
several students on one instructional materials lesson as 
permitted, and educational contests, such as spelling bees. 
Further, this invention is equally adaptable to other forms 

20 of materials that function in the framework of Fig. i, in 

particular interfacing to the agent software as indicated by 
arrow 111, and that are useful in a particular application of 
this invention. For example, materials appropriate for child 
care contexts can differ from the above three classes by, 

25 perhaps, having different paradigms of interactivity. 

The structure and course of interactions 103 between the 
student and the materials is preferably governed by paradigms 
of educational psychology and sound educational practice, 
such as are described in the exemplary reference Englemann et 

30 al., 1982, Theory of instruction; principles and 

appj^icatiops. New York: Irvington Publisher. At the most 
immediate level, for example, during homework or instruction, 
student 101 can make requests and receives responses from 
^^^©rials engine 102 and, in turn, materials engine 102 can 

35 make requests and receive responses from student loi. The 
materials engine can adjust its sequence of. presentation in 
response to student responses. At a next level, the requests 
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and responses exchanged between the student and the materials 
engine can follow several patterns known in the arts of 
computer based instruction and which ^ for example, include 
the following. First, the student can respond to questions 
5 presented by the materials engine, and in the course of 
responding, can ask for advice or hints, the use of a tool 
such as a calculator, or other relevant assistance. Second, 
the student can advance to the next item, lesson, or unit 
upon successful completion of the present item, lesson, or 

10 unit. Third, in case of error, the student can request, or 
automatically be presented with, appropriate repeat, review, 
or remediation materials. Finally, at a higher level these 
patterns of interactions can be analyzed to provide more 
adaptive responses from the system. 

15 Teachers and administrators 106 also interact with 

Ttnaterials engine 102 for several purposes, as represented by 
arrow 107. For example, teachers can customize existing 
materials by adding additional items, modifying existing 
items, altering the order of item presentation, changing the 

20 notations (see Infra.) governing agent interaction, and so 
forth. A teacher can create particular instances of 
materials suitable for one class, a group, or even one 
student. Further, although reporting of student progress 
preferably occxirs by other means, in alternative embodiments 

2 5 the materials engine can directly report student progress to 
teachers and administrators. For example, this can be done 
by entering notations that generate messages for the 
teachers • 

Also, instructional designers can create, or "author," 
30 materials for use in this invention. Such materials can be 
original or can be derived from existing textbooks, workbooks 
or media material. They can employ standardized curricula, 
pretests such as criterion tests, post-tests, and 
standardized texts. Authoring instructional materials in a 
35 course suitable for interactive instruction typically 
comprises several steps, including decisions about the 
objects to display to the student, the sequencing of these 
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Objects, and the interactions with the agent. The first step 
is the selection of objects which carry the education content 
for presentation to a student. Objects can include visual 
display items, such as text, graphics, animation or movies, 
5 audible display items, such as voice, audio and so forth. 
They can include input items known in the computer arts, such 
as buttons to select, selections to chose from, text to 
enter, hypertext and hypermedia links, functions to perform 
with student input, and so forth. The second step is the 

10 selection of the sequencing logic for the ordered display of 
the objects to the student and the educationally appropriate 
reaction to student requests and responses. The sequencing 
logic can reference instructional controls set by agent 
software 108, such as a command to increase example density, 

15 and preferably is chosen in light of principles of 

educational psychology and practice as detailed above- The 
third step is the specification of interactions with a 
student's agent or virtual tutor, a key component of the ABI 
system. This specification is made by augmenting the 

20 sequencing logic with "notations," which are referenced, 
called, or executed by the sequencing logic during object 
presentation and that communicate with the agent, in a 
preferred embodiment by exchanging messages. In the ABI 
system, the agent builds an adaptive model of its student's 

25 pedagogic characteristics, in other words the student's 

cognitive styles, by monitoring the course of the student's 
interactive instruction. The notations are the means for 
this monitoring. Finally, in order to make materials 
available, the authored materials are indexed and stored in 

30 the files of the ABI system, preferably on materials server 
systems • 

ABI authoring tools differ from authoring conventional 
instructional materials in that notations are present in 
these materials to enable the agent software to update the 
35 Student data object, to monitor and modify the instruction, 
student's use of a tool, or a communication task. ABI 
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authoring tools support and facilitate the conversion of 
existing mater ia Is . to the ABI instructional format. 

As indicated above, an agent unique to each student loi 
is important in the ABI system. A student's agent is 
5 comprised of agent software 108 in conjunction with a student 
data object 109 unique to each student. As the agent 
software monitors its student's instruction, it builds an 
adaptive model of its student in student data object 109. 
Guided by this model, agent software 108 acts, first, to 
10 manage or control the student's instruction, and second, to 
directly guide the student in order that the total ABI system 
can present education to each student in an optimal fashion 
best adapted to the student's evolving abilities, skills, and 
preferences. In other words, the agent becomes a virtual 
15 tutor by acting as a student's personal and individualized 
tutor. First, the agent manages or controls instruction of 
student lOl by directly controlling materials engine 102 in 
its presentation of materials data 104, 114, and lis through 
interaction with the materials engine, as represented by 
2 0 arrow ill. The agent preferably manages or controls the 
materials engine in two manners, synchronous with materials 
data presentation, such as when the materials engine 
encounters an appropriate notation in the data, makes an 
agent request, and waits for an agent response, and 
25 asynchronous with the presentation, such as when the agent 
software adjusts control parameters which the materials 
engine references at decision points. Examples of 
synchronous control are an instructional material asking the 
agent for permission to allow the student to use a tool, to 
30 receive a hint, or to be given remediation, or a 

communications material asking the agent for permission to 
permit the student to engage in a particular type of 
communication with certain other students. An example of 
asynchronous control is the agent setting of pedagogic 
35 parameters, such as coaching parameters that the materials 
engine uses to adjust its presentation, according to the 
pedagogic characteristics of the student. Exemplary coaching 
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parameters include the time pacing of exercises , the new 
concept seeding rate and the density of examples. In this 
manner, the materials can present interactive instruction 
according to optimal values of the pedagogic characteristics 
S or cognitive styles of each student as determined from the 
agent's observation of its student. 

Second, agent software 108 directly guides the student 
by exchanging communication with the student, as represented 
by arrow 112. Student communication also preferably occurs 
10 in two manners, synchronously, in which the student directly 
makes meta-requests of the agent tutor and receives meta- 
responses and second, asynchronously, in which the agent 
tutor itself generates a meta-response in response to some 
instructional event. Herein, requests and responses are 
15 prefixed with "meta" when they are exchanged directly with 
the agent. Meta-recjuests include student questions to the 
agent - for example: How am I doing? What should I do next? 
Could you say that another way? - or student requests - for 
example: I need a hint; I need help. The agent responds to 
2 0 each student question or request. Agent meta-responses can 
be generated, for example, when the student takes too long to 
complete an exercise, when the student makes a series of 
repeated errors, or when the student achieves good 
performance. Agent meta-responses can be drawn from such 
25 categories as reminders, encouragements, reinforcements, 
paraphrases, jokes, progress summaries, and so forth. 

Communication with the agent, represented by arrow 112, 
include direct student meta-requests that generate agent 
meta-responses. Other commxinications derive from 
30 instructional event messages generated and communicated by 
augmented notations in materials 104, 114, and 115. An event 
received by the agent can generate no meta-response at all or 
alternatively can generate an asynchronous type agent meta- 
response. At educationally significant points, as the 
35 materials sequencing logic presents display objects to the 
student and receives inputs from the student, the materials 
data author places one or more notations. When these 

- 23 - 



BNSOOCID: <WO ^9744766A1 _!_> 



wo 97/44766 



PCT/US97/0868S 



notation are referenced, called, or executed, important 
variables and parameters educationally relevant at this 
significant point are gathered into a message, along with an 
indication of the type of the educational event- These 
5 messages are events which are then sent to the agent. For 
example, an educationally significant point is the beginning 
of a new instructional sequence. The corresponding event 
message can include an indication of the topic to be covered, 
the expected level of difficulty, the expected time to 

XO complete, and the educational paradigm adopted. Another 
educationally significant point is the receipt of a wrong 
answer. In response, the materials can generate several 
messages: a first message can include the time required to 
make the answer, an indication of the type of error, and an 

15 indication of whether the answer is completely wrong or only 
a near miss; a second message can include text parameters 
("say-it" type message) if the agent chooses to make a 
specific text or spoken comment about the error; finally, a 
third message can include the screen location best 

2 0 representing the error ("point-it" type message) to use if 
the on-screen agent chooses to point to the error or move to 
the location of the error. Another educationally significant 
point can be a long delay in receiving the next student 
input, at which point the materials engine can send an 

2 5 asynchronous message indicating the time elapsed. Tools data 

115 generate events similar to messages from instructional 
materials. Conuaunications materials 104 can generate events 
recording a communication request or an initiation of 
communication with certain other students for a certain task. 
30 In the case of shared work on one materials, communications 
materials can generate events recording how this student in 
progressing with the shared materials; in the case of a 
contest such as a spelling bee, events recording how this 
student is progressing in the contest with respect to other 

3 5 contestants . 

In addition, in a preferred embodiment agent software 
108 also receives messages describing the progress of the 
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Student through specific insrructional materials. For 
example, in the case of mathematics materials, such messages 
can include information that the student is making errors in 
problems requiring finding common denominators. These event 
5 message should preferably all information that can be of 
interest to teachers and administrators for tracking student 
progress and tracking course adequacy. 

It is important that communication between the student 
and the agent be engaging. Agent communication preferably 

10 utilizes all the modalities of input and output available in 
a particular implementation of this invention, including 
text, audio displays such as voice and sound, and video 
displays such as graphics, animation, and realistic movie 
clips. For example, in the case of a communication triggered 

15 by good performance, the agent can select the display of 
sound and video clips, from a data snips library, that the 
student finds pleasing. The agent can further make reward 
graphics available on the student's screen for a period of 
time. On the other hand, in the case of error the agent can 

20 point to the screen location of the error. 

Further, it is highly preferable that the on-screen . 
agent can assume various display personae during student 
communication. Herein, persona means the effect conveyed to 
the student of the combined and coherent presentation of 

25 multiple display modalities to emulate a particular, 

apparently living, personality. For example, in the case of 
elementary education, this can be the selection of tone and 
animation to emulate a pleasant animal or a known cartoon 
character. In some cases, characteristics of the display 

3 0 persona can be selectable by the student according to the 
student's preferences. In other cases, the personae can be 
specified by the instructional materials, the teacher or the 
administrator overriding student persona preferences. 
Personae for an elementary school student can be selected 

35 from well-known cartoon characters and can perhaps be called 
"Study Buddies™." Persona for commercial or corporate 
education can be adapted to the organizational ethos and can 
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perhaps be called a "Concept Coach." Presentations for 
intermediate levels can be called an Electronic Learning 
Friend ("ELF"). Even though the ABI system through its 
network, software and database acts as the student's virtual 
5 tutor, from the elementary school students point of view, the 
"Study Buddies^" are his/her personal tutor. Realism in 
voice, gestures and movement reinforce this relationship. 

Agent software 108 in the ABI system builds an adapting 
pedagogic or cognitive model of its student in student data 
10 object 109. This model is independent of specific materials. 
Event messages to the agent software from the materials 
engine preferably include the information from which this 
model is built. In general, event messages must include such 
content as is necessary to describe and parametrize the 
15 pedagogic or cognitive style models adopted by the materials 
in an implementation of the ABI system. 

In a preferred embodiment, the student data object 109 
includes such data about a student necessary to the agent's 
evolving pedagogic or cognitive student model. Further, it 

2 0 preferably also includes data concerning the student's 

current state and recent performance in the instructional 
materials being presented to the student. This latter data 
is interpretable by both the agent and the instructional 
materials. Finally, the student data object can include 
25 private scratch-pad areas for the sole use of instructional 
materials. Data in the student data object is available for 
access during student sessions and is safely stored between 
sessions. 

The student data object is preferably securely stored on 

3 0 portable media in the physical control of the student. A 

student "plugs- in" such a portable student data object stored 
on portable media as part of accessing an ABI system. 
Preferably, this portable medium is a smart card, or similar 
device, which securely couples portable storage and 
35 processing in order to shield the actual data and, instead, 
present a message-based interface to the student data object. 
Alternatively, the student data object iran be centrally 
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Stored on school computers and downloaded to the student's 
computer when the student access an ABI system. in this 
case, the student data objects are preferably stored in an 
object oriented database, although this data can be 
S advantageously stored in standard relational databases. As 
depicted by arrow iio, the student data object is referenced 
by the agent in order to generate its actions and is updated 
by the agent as it processes events and student meta- 
requests. 

10 Especially in the primary and secondary school context, 

the school can need to gather educational progress data 
pursuant to its data-gathering and privacy policies. In 
cases where no single school is present, as in the case of 
multiple, dispersed education providers, no such school - 

15 student data is generated by the agent. However, such 

school-student data 116, when present, can be forwarded to 
central school computers by agent 108. Also, school-student 
data can contain guidance to the student's agent, such as the 
school schedule. Accordingly, school-student data is 

2 0 accessed not only by the agent software 108, but also by 
teachers and administrators 106. As depicted by arrow 113, 
school-student data is referenced by teachers and 
administrators in order to track the student progress and to 
generate reports concerning the students and the materials in 

25 the ABI system. Teachers also update the school-student data 
to enter schedule information for the student's assignments, 
which can be referenced and used by the student's agent. 
Administrators update this data in order to enter the courses 
and materials the student must master and specify standards 

30 and criteria the student must meet. 

In an alternative embodiment, however, these various 
data can be combined. Student data object 109, whether 
portable or centrally stored, can contain both pedagogic and 
cognitive model data and educational progress data, the 

35 school-student data 116. Such a combination is particularly 
advantageous in the case of central storage of the student 
data object. 
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Exanplarv laplM ^ntina struetura 
Figs. 2A and 2B illustrate an exemplary preferred 
structure implementing the principal conceptual and 
functional components of the ABI system as illustrated in 
5 Fig. 1. A preferred implementation of this invention is 

based on a plurality of computers interconnected by a network 
and, preferably, a plurality of portable storage media for 
the student data objects. Preferably, although all system 
functions can be performed on all interconnected computers, 
10 certain computers are specialized for certain functions, such 
as student client systems for providing student access or 
system servers for storing system software resources. 
Therefore, an exemplary preferred ABI system includes one or 
more student client systems 201, at which student 202 
15 receives instructional presentations including homework. 

Other student clients are generally indicated at 203 and can 
be located at school, at home, or at the office. The system 
further includes one or more servers 2 04, at which teachers 
and administrators 205 gain access to the system. 
20 Alternatively, they can access the ABI system at their own 
client computer systems. 

These components are interconnected by a network which 
consists of transmission medium 206 and local attachments 
207. Although the network illustrated in Fig. 2A is of a 
25 bus-type configuration, as in a local area network perhaps 
using collision detection or token passing protocols or both, 
this invention is adaptable to all forms of networks which 
support adequate transmission protocols, such as those that 
are functionally similar to the TCP/IP protocol suite, and as 
30 appropriate, ATM technology. Thus, the invention is 
adaptable to networks constructed from switched or non- 
switched links to a central server, which can be configured 
of several LAN attached server systems, to networks including 
CATV cable or optical links, to networks including radio 
35 links either terrestrial or satellite, and to public or 

private packet switching networks. In the near future, for 
the majority of students, local attachment 207 is expected to 
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remain a lower speed access over the switched, public 
telephone network. In the more distant future, such lower 
speed local attachments may possibly be replaced by a higher 
speed access using currently developing high-bandwidth 
5 technologies. 

A preferred public packet switching network to which 
this invention is adaptable is the Internet. ABI system 
nodes can be linked to the Internet in any convenient manner 
known to those skilled in the art and can include security 

10 boundaries (known a firewalls) to prevent unauthorized 

access. Alternatively, an ABI system implementation can be 
an "intranet," that is a network constructed according to 
TCP/IP protocols but using private links of the types 
enumerated above, and as appropriate, ATM technologies. 

15 In more detail, student client system 201 includes 

memory 208, which is RAM type real memory or a virtual memory 
based on RAM type memory and a backing store. Preferably, to 
reduce size and cost, the client system has no permanent disk 
storage. When available, a preferable student client can be 

20 a low cost network computer ("NC") . A NC is a computer with 
processor, RAM, and network interfaces sufficient to access 
intranets or the Internet. A NC is preferable in cases when 
high-bandwidth network access is available. In cases of low- 
bandwidth network access, the client system can have one or 

25 more disc drives 209 which can be used as a pre-fetch buffer 
or a read-only cache. The disks preferably are magnetic, 
although in a less preferable embodiment they can also 
include CDROMs. This optional capability serves to enhance 
communications efficiency in cases where the network has 

30 relatively low bandwidth. Large files can be downloaded in 
advance of a student session or the student client can cache 
read-only data across sessions obviating the need for 
downloading such files. Such caching requires the operating 
system components to maintain some form of version control of 

3 5 the read-only data. 

Preferably, an ABI system also includes portable storage 
media 240 storing a portable student object. Each student 
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has physical possession of such a media which contains the 
student data object for the student. The data on portable 
media 240 is accessed by client system 201 across an 
appropriate physical link 241. Advantageously, portable 
5 media 24 0 contains both storage and processing capabilities. 
The contained processing capability responds only to inquiry 
and update methods for data elements of the student data 
object; no methods or data need be loaded into RAM memory 
224. Thereby, method execution is shielded from client 

10 system 2 01, and the actual data structures of the student 
data object are protected from accidental or malicious 
alteration or damage by any foreign software that might be on 
client system 201. in cases where adequate processing 
capabilities are present, such a portable media can implement 

15 many of methods of the student data object, appearing 
essentially as a "pluggable object." Illustrative such 
portable media can be a smart card or a PCMCIA card with 
flash RAM and a microprocessor. Such portable media also are 
more platform independent than are alternative media. 

20 Alternatively, portable media 240 can be a portable hard 

disc packaged for access through a parallel or serial port of 
client system 201. The portable media can also be a standard 
floppy disc. In this alternative, student data object 
methods can be stored as executable modules and object data 

2 5 structures can be stored in a securely encrypted format to 
prevent unauthorized alteration of data values or data 
structures. Student data object access from such portable 
media begins with the loading and executing of appropriate 
executable methods by client system 201. These methods then 

30 fetch and decrypt requested data elements into RAM memory 224 
for reference or for update. in this case, only the data is 
shielded from client system 2 01. This invention is adaptable 
to other portable media for the student data object, such as 
one portable client computer for, and in the possession of, 

35 one student which has that student's data object as a 
permanently mounted component • 
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Alternatively, in cases where portable media are deemed 
inappropriate, the student data object can be stored between 
student sessions on a server and downloaded when a student 
accesses an ABI system at a particular client, 
5 According to all of these alternatives, a student can 

access ABI system services from any available client system 
at any time. 

The student interacts with a client system using any 
appropriate interactive input/output ,{ "I /o"« ) modes 210. For 

10 input, standard devices include pointing devices, such as 
mouse 211 or a track-ball, age appropriate keyboards, 
optionally speech recognition, and so forth. Speech 
recognition will permit brief conversations with the "Study 
Buddies^," or other personae, in limited areas. The 

15 invention is adaptable to special input devices appropriate 
for particular groups, such as the handicapped, and to 
devices yet to be constructed. Virtual reality ("VR") 
interface devices such as VR gloves and VR display helmets 
can have immediate applications for special needs students, 

20 For output, preferable devices include computer display 212, 
for displaying objects such as text, graphics, animation, and 
video, and audio output devices for voice and sound clips. 
The audio and voice can be constructed from data snips stored 
as digitized sound files in libraries. Alternatively, voice 

25 can be synthesized from text. The invention is also 

adaptable to special output devices for special classes of 
students, e.g. the handicapped, and to new devices being 
developed. 

Fig. 2A also shows an exemplary screen layout for a 
30 student client that exemplifies the principal functions of 
this invention. The screen is preferably partitioned so that 
principal components of this invention are displayed; and 
important student actions are represented by icons or 
buttons. Thus, the screen includes materials and tools area 
35 220 to the left, agent area 215 to the right, and a system 
toolbar 218, which includes a student customization area 221 
at the top. The size of the screen partitions illustrated 
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preferably change from time to time in response to student 
customization or display requirements. In particular, either 
the materials area or the agent area can enlarge, perhaps up 
to the entire screen as needed. Materials area 220 is for 
5 the instructional materials, tools, and communication 
materials to present visual display objects and for these 
components to receive interactive input. This area is 
further subdivided into display region 213 and a materials 
specific toolbar 214. On-screen agent area 215 is for the 

10 on-screen agent to receive meta-requests and to display 
synchronous and asynchronous meta-responses. Also 
illustrated is an exemplary on-screen agent consisting of a 
single persona 216 and a meta-request icon 217. The persona 
can move to other screen areas as required. Both these 

15 components are illustrated as appropriate for elementary 

education. Other visual appearances would be appropriate in 
other situations. The system area at top includes toolbar 
218 for selecting particular available system components. In 
particular, always available on this toolbar are selection 

20 icons 219 for the calendar and scheduling tool. Part 221 of 
the system area can be reserved for student customization, 
for example for the display of reward graphics "given'' to the 
student by the agent or virtual tutor. 

Functionally illustrated in Fig. 2A is an exemplary 

25 memory organization of a student client system when a session 
is in progress with materials being presented. Layer 22 2 
comprises operating software and network communications. 
This software provides, among other services, support for I/O 
devices attached to the client, a file system with cache 

30 control, lower level network protocols, such as TCP/IP and 
ATM, and higher-level network protocols, such as HTTP V2.0, 
Executive software layer 22 3 provides basic shared ABI 
system capabilities. The executive software verifies student 
identity and access authority, establishes communications 

35 sessions with the system servers as required during client 
start-up, downloads instructional materials scheduled for 
this student, and download executable S£>ftware required from 
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the systems servers as-needed. in the preferred embodiment 
using portable media, the executive software verifies that 
the contained student data object belongs to the identified 
student and accesses the methods of the student data object 
5 upon request, primarily, from agent software 225, In an 
embodiment with centrally stored student data objects, the 
executive software downloads the identified student's data 
object from the student database. The instructional material 
and the software are read-only and are- not changed during a 
10 session. The student model is updated by the agent during 
the student session and modified parts are stored back in 
portable media 24 0 or uploaded to the student database on a 
server. Such downloading can utilize higher level network 
transfer protocols, or alternatively, directly use lower 
15 level network protocols. 

Fig. 2A illustrates that agent software 225, optional 
components of student data object 22 6, and certain 
instructional materials software 224 have already been 
downloaded. The materials are displaying objects in screen 
20 area 220, and as indicated by arrow 227 are forwarding events 
to the agent and receiving management or controls from the 
agent. The agent is displaying its persona (e) in screen area 
215, interacting with the materials, as represented by arrow 
227, and is referencing and updating data in student data 
25 model 226, as represented by arrow 228. 

Arrow 228 represents different access methods depending 
on the embodiment of the student data object. In the case of 
preferred portable media with processing and storage, no 
components of the student data object are loaded, in memory. 
30 Then, arrow 228 represents direct message exchanges, or 
method calls, to portable device 240 made across physical 
link 241. In the case of portable media such as hard disks 
or floppy disks, arrow 228 represents method calls to student 
data object methods loaded into memory 22 6. These methods 
35 can reference or load into memory additional necessary data 
components of the student data object. In the case of a 
downloaded student data object, arrow 22 8 merely represents 
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memory accesses to this data. Finally, in the preferred 
embodiment where the student data is separated into the 
student data object and into school-student data, arrow 22 8 
further represents references and updates to such school- 
5 student data, preferably on school servers. Such updates can 
include school-related educational progress data. 

Finally, the student client system further includes 
standard components not shown, such as a microprocessor and 
input/output interfaces. Alternative implementations of the 

XO student client system are within the scope of this invention. 
In particular, student client system function can reside on a 
server system. Further, the client system can be implemented 
as two machines, wherein a first machine performs 
substantially all the computations and a second machine is 

15 for student interaction and sends student input and receives 
display instructions from the first machine. 

The ABI system further includes one or more server 
systems 204 of Fig. 2B with sufficient large capacity discs 
230 for storing all school-student data, all instructional 

20 materials, and all software used in the system. Optionally, 
the servers can also store student data objects in an object- 
oriented database. The network is used to distribute 
software, instructional materials, and school-student data 
from these servers. In alternative embodiments, there can be 

2 5 more than one server with software and data component storage 

divided as convenient across the servers. In a further and 
less preferred embodiment, the server can be a central host 
system. In comparison to the client systems, servers 
preferably have increased performance and higher speed 

3 0 network connections 231 in order to make this stored data 

quickly available to the one or more student client systems. 
Access to school-student data allows teachers and 
administrators to track student performance by class, grade, 
subject, school and so forth, 
35 Importantly, certain server systems can also perform a 

registry function. In cases where multiple schools or 
multiple education providers can be accessed by a student, it 
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is important to ensure integrity of that student's data 
object. In these cases, before allowing dovmloaded software 
to access the student data object, either indirectly through 
the agent, or, perhaps, directly, such software can be 
5 validated as being both a correctly functioning component of 
an ABI system and a correctly identified component. in 
particular, the latter check prevents foreign software from 
masquerading as legitimate ABI system software. The 
information for these checks can be stored as a registry 
10 function in certain common server systems. See, infra., 
section 5.2.4. 

The server systems are preferably configured as shown in 
Fig. 2B and are loaded with software 2 32 providing the 
following function. First, there is operating system, 
15 network services, and file server layer 233. In addition to 
the functions of layer 222 in the client system, layer 233 
also provides a file server facility including file backup 
and version control. System manger 234 includes facilities 
for access control, authenticating student access requests 

20 and limiting file access to authorized users. For example, 
students can be limited to only their personal files; parents 
to their children's files and curricular related data; 
teachers to files and school-student data for their classes; 
while certain administrators have unlimited access. The 

25 system manager can also maintain any necessary system logs. 
Instructional materials databases 240 and directories of 
executable software also reside on the server systems. when 
group communications is in use, agents and communication 
tasks monitoring the groups can reside on the servers. The 

30 servers also contain areas 237 for administrative data and 
for reports and report software of interest to the 
administrative staff. The servers also contain teacher areas 
23 8 for data and report software of interest to teachers. 

Finally, the servers contain student database 235. This 

35 database preferably includes school-student data, which is a 
source of reports for teachers and administrators and of data 
by which these staffs schedule student courses and 
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assignments. Preferably, student data objects, including 
data representing the pedagogic model of the student used by 
the agent software, are stored in portable media 240. in 
this embodiment, the servers preferably provide an area for 
5 secure backup storage for these portable data objects . 

Optionally, student data objects can be stored in an object- 
oriented extension to student database 2 35. 

Further, certain instructional materials can be made 
available specifically for the teaching staff, along with an 

10 individualized teacher agent acting as a virtual tutor for 
the teachers. The facilities of the ABI system can be used 
to simplify teacher familiarization and system training. For 
example, such teacher training can be user instruction in the 
ABI system itself, or can be teacher versions of student 

15 materials designed to assist the teacher in his/her use and 
customization (in the nature of today's teacher versions of 
textbooks). Further, there can be teacher specific tools, 
for example, to assist in generating student reports and 
class management. In this case the server system contains 

2 0 special instructional materials for performing this 

instruction. Associated teacher data objects are implemented 
and stored similarly to student data objects. Client systems 
for teacher access have agents unique to individual teachers. 
Alternatively, the flexible server structure of the 

2 5 system permits administrative and teaching staff to perfozrm 

their specific tasks on any computer system with enough 
computing resources to support these tasks. As for student 
access, necessary system components are downloaded from 
servers to these temporary client systems. Thereby, these 

3 0 personnel are not limited to sessions on server systems. 

Further, materials authoring can be done on server systems, 
client systems, or on separate systems not interconnected 
with a given ABI system. To make authored materials 
available, they are transferred to and indexed in the 
3 5 appropriate server system databases. 

Alternative implementations of the functions described 
for the student client systems and the server systems are 
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also within the scope of this invention. For example, it i 
known to those of skill in the art that by the use of vario 
technologies, such as remote procedure calls or messaging, 
the functions pictured here as grouped together and on one 
5 system can be divided and distributed if needed, 

5.2. BvsteM Hardvare and operating Sof tvara 

The system elements illustrated in Figs. 2A and 2B can 
be constructed on standard hardware platforms meeting the 
10 requirements discussed in this section. 



5*2. 1. The ABI Svstaa Network 

It is important that students or teachers be able to 
access an ABI system from any adequate client system anywhere 

15 at the school, at home, and when away from home, as in 
distance learning. Portable student data objects and 
networks permit students and teachers to participate at more 
than one school, and, further, permit delivery of homework 
and instruction to remote locations. In each case the client 

20 system must access instructional materials and ABI software 
from the network. 

In various embodiments, a student can access one ABI 
system, as might be the case for a student enrolled at one 
elementary school. Alternatively, a student can have access 

25 to multiple ABI systems or multiple providers of ABI-system 
compliant instructional materials. 

Access to all system components is typically provided 
from ABI system servers attached to the network. Preferably 
i^«a^"Only software and read-only instructional materials are 

30 provided from server systems. The advantages over local 
storage of these elements at client systems include 
simplification of version control and of access control. The 
student is freed from insuring the client system accessed has 
all the necessary software and manually transferring if not. 

35 Also server storage provides greatly reduced client system 

storage requirements, in view of the large amount of software 
and instructional materials in ABI systems. However, in 
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systems with limited communication bandwidth, it can be 
preferable to cache recently used software and instructional 
materials in order to shorten response times. 

Networks suitable for an ABI system can be of any 
5 configuration and protocol that meets the system's client- 
server communication and object transfer requirements, where 
the client is either a PC or a network computer ("NC"). 
Suitable networks can be private or public. The preferred 
ABI network in the case of PC clients, according to current 

10 technologies, is an intranet running the TCP/IP protocol 

suite and utilizing ATM technologies as appropriate. In one 
implementation, utilizing a single large server, such a 
server can be directly connected to the intranet or Internet. 
In another implementation, utilizing several smaller servers, 

15 these servers can be connected into a cluster by a LAN, 

preferably an Ethernet LAN. In this case, the ABI network is 
built from a local Ethernet LAN with remote connections to 
telephone lines using 28.8 Kbps modems, or other network 
links. In an alternate implementation, the network can by 

2 0 visible to the public Internet if adequate security systems 
(firewalls) prevent unauthorized access. This can make wider 
access available at lower cost than by switched telephone 
remote connections . 

The ABI LAN can be further connected to other ABI LAN's 

2 5 and to other networks, such as the Internet, by standard use 
of routers and gateways. An exemplary protocol for an ABI 
System network is TCP/IP supplemented with a client-server 
protocol such as HTTP and object transfer protocol, such as a 
multi-format mail protocol. ATM technologies are used as 

30 appropriate. Remote clients are expected to be indirectly 
connected to the ABI network. These connections can consist 
of routing over a public network or direct dial-in 
connections. These connections can be of either low-speed or 
high speed. The ABI system can be implemented on networks 

35 such as §Home (Mountain View, CA) , which use a combination of 
ATM, cable modems, TCP/IP and other technologies. The ABI 
system can also be hierarchically configured on new network 
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topologies for distance learning in areas with limited 
communications infrastructure. Primary central servers with 
ABI software and instructional materials communicate with 
remote secondary servers over broadband satellite 
5 communication systems. Student clients connect to the local 
secondary servers through wired or wireless means. 

Considerations governing the choice of client hardware 
10 and operation software are described next and are followed by 
a preferred client node system according to today's 
technology. These consideration depend on the content of an 
ABI system and the facilities of technology. As the content 
of a system changes and as technology evolves these 
15 consideration dictate that the preferred system will change. 
The client hardware consists of client input/output 
("I/O"), client CPU and memory, and client network access. 
Turning to the client I/O reguirement, standard input devices 
such as keyboard and mouse, or other pointing device, are 
20 preferable. The client system also includes interfaces to a 
portable student data object. Such interface can 
alternatively include a smart card access device, PCMCIA 
ports, standard parallel or serial ports, or floppy disk 
drives. Color graphics output capability adequate to support 

2 5 partial screen animations is preferred. Sound generation and 

output are preferable on ABI client systems. Text-to-speech 
conversion can be done either in software or in hardware. 
When economically available, full video capability, for 
example by providing video decompression hardware such as 

3 0 MPEG decoders, and speech recognition, for example with 

hardware assists, are also preferable. 

The ABI system is also adaptable to special I/O devices 
appropriate to special student groups, such as the very young 
or the handicapped. These include, for example, simplified 
35 keyboards, touch panels, VR devices, and so forth. 

Client memory must be sufficient to contain resident 
operating system components, resident ABI executive software, 
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and dynamically loaded segments of the student data object, 
instructional materials, and code. High performance CPU's' 
together with high performance graphics hardware and memory 
is preferable to enable more advanced presentation effects. 
5 In embodiments where part or all of the ABI software is 

implemented in special languages, hardware or software 
assists for these language are preferable. For example, 
where such a special language is JAVA" (Sun Microsystems, 
Mountain View, CA) , JAVA™ chips, which enhance performance of 
XO the JAVA™ interpreter, are preferable. The ABI client node 
in certain embodiments can be a JAVA"* enhanced network 
appliance adapted to Internet communication access and the 
HTTP V2.0, or equivalent client-server protocol. 

It is preferable to eliminate the need for permanent 
15 data storage devices, such as hard disc drives, by 

downloading all client software for each session. This 
allows the use of low cost 'network appliances' as student 
client computers. However, for those clients with lower 
speed network access asynchronous downloading can be used. 
2 0 Storage devices such as hard drives can be preferable for 
pre-fetching or caching of read-only software components in 
order to reduce start up time. To download 4 megabytes of 
software using, for example, a 28.8 Kbps modem takes over 20 
minutes. In the case of caching on student client, standard 
25 version control methods known in the art are necessary to 
ensure that only up-to-date software and materials data are 
used. If an element is found to be out-of-date by querying a 
server, the current version is downloaded. 

The portable media for the preferred embodiment of the 
30 student data object includes devices with storage and 

processing capabilities, such as smart cards and PCMCIA cards 
with flash RAM and a microprocessor. . Alternatively, portable 
hard or floppy drives as are coiimonly available can be used 
to store the student data object. Typically, this data 
35 object, including accompanying access methods, is 

approximately 100 KB, and preferably less than approximately 
1 MB. In the case in which the student data object is stored 
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on server computer, it is not be cached between sessions so 
that it can be accessed from other client systems* 

Client communication hardware can be adapted for either 
local or remote attachment. Local access requires network 
5 access hardware; remote access requires a communication 
capability. This invention is adaptable to lower speed 
access over switched telephone line services, preferably 
using 28.8 Kbps modems or ISDN interfaces (64 or 128 Kbps) . 
These bandwidths are adequate for sessions with materials 

10 using only voice and limited animations. Pre-f etching and 
caching can be required to make fullest use of other 
materials at this bandwidth. This invention is also 
adaptable to high speed access over any available high speed 
links, such as Tl (1.5 Mbps) , T3 , ADSL telephone lines, or 

15 cable modems (several Mbps) , or other means of high speed 
access. These bandwidths permit full access to materials 
without limitation. If economically available, high speed 
access is preferred. With greater communications bandwidth, 
the on-screen agent can appear more life-like. 

2 0 Standard client software includes an operating system 

and communication software. The operating system preferably 
has interfaces to client I/O devices including communications 
capability and network access, such as a TCP/IP stack. ATM 
interfaces are present if necessary. Preferably, it also has 

25 means for establishing sessions with servers, for providing 
file server services, and means for security as specified 
shortly. 

The implementation language or languages of this 
invention preferably have several features related to the 

30 implementability, maintainability, and extensibility of ABI. 
The implementation language preferably provides a degree of 
modularity similar to that provided by object-oriented 
programming languages. It preferably provides means of 
dynamically loading across a network and executing additional 

35 software segments during program execution. It preferably 
provides means of accessing all input devices and of 
controlling the output devices in a high level display object 
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fashion. It preferably provides a threaded or 
multiprocessing capability. Less preferably, this invention 
can be implemented in any computer language, including 
assembly language. 
5 In view of these needs and in view of the current 

technology, a currently preferred client system is a IBM type 
PC with a Pentium™ 120 Mhz processor, 16 MB memory, a l GB 
disk drive, a Soundblaster compatible sound card with 
speakers, a medium performance graphics card such as a 

10 Diamond Stealth card with 2 MB of graphics memory, an 

Ethernet card or a communication card and a 28.8 Kbps modem, 
and standard keyboard and pointing device such as mouse. The 
operating system is Windows™ 95 with network services 
provided by a World wide Web browser equivalent to Netscape 

IS 2.0 or better and capable of running Java™ applets. Java™ 
together with standard system and graphics classes is the 
implementation language - 



5.2.3. The flervy ay ***^**- 

20 The functions of server systems of this invention 

include storage of databases of executable software elements, 
of school-student data, if any, and of instructional 
materials. Optionally, it can also store student data 
objects. The latter elements consist of heterogeneous and 

25 structured elements. These elements can be stored in a 

relational database such as supplied by the Oracle Corp. or 
the Sybase Corp.; they can be stored as specialized data 
files; or they can be stored in an object-oriented database 
system such as Objectstore (Object Design Inc., Burlington, 

3 0 MA) . The operating system of the server nodes must support 
whatever database systems are selected as well as network and 
application server software to access the databases. 
Application database server software of this invention 
preferably provides database access and version control and 

35 downloads database elements on client request. 

The preferred server hardware and software can vary 
widely depending on the number of clients to be 
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Simultaneously served. This number can vary from 20 at one 
school to more than 5000 across an entire school system. The 
number of servers and database distribution across a server 
cluster can be adjusted by means known in the art to satisfy 
5 projected peak loads. A suitable medium performance server 
system can be configured on a high end INTEL Pentium or DEC 
Alpha system with adequate memory and disk space. Windows'** 
KT is an adequate server operating system, and Internet 
server software similar to that from Netscape is adequate for 

10 network access. The preferred database is an object oriented 
database such as ObjectStore. In this embodiment, 
application database access uses a common gateway interface 
(«CGI") program also providing database access and version 
control. The CGI access program can be implemented in C++, a 

15 suitable object oriented programming language capable of 
accessing interfaces to ObjectStore databases. 

5.2,4. ABI 8vste« Saeurity 

Security and access control present additional client, 

20 server, and portable media requirements which are importantly 
part of an implementation of this invention. Security and 
access control can be maintained by careful selection of 
management policies, security software, and security 
hardware. These elements are described in this section in the 

25 order of authorizing and controlling access, operating system 
and network security requirements, implementation language 
issues, and portable student data object security. 

In a preferred embodiment, the primary means for 
authorizing and controlling access are passwords. System 

30 management of passwords preferably includes ensuring that 
user passwords are secure, not easily guessed, and are 
periodically changed. This invention is also adaptable to 
any other means of access control, including for example, 
passive and active identification cards and recognition of 

35 certain personal characteristics, such as voice. Access 
protection can be preferably provided by limiting access to 
system resources - database and file - based on a user's 
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password. For example, access protiection can be implemented 
in the CGI application access programs. 

The operating system in clients and servers of this 
invention is preferably of a tested security level. This 
5 base security can be enhanced by a variety of techniques and 
tools that can provide increased levels of security for 
additional investments. Such techniques and tools include 
firewall machines, that is dedicated network gateways that 
filter incoming and outgoing packets .according to content 

10 criteria, and monitoring software, such as tripwires, that 
observe system events for suspicious combinations. Further, 
encryption can help protect sensitive and valuable data from 
illegitimate access by those without the key. Encryption in 
hardware and software can be provided according to methods 

X5 known in the art, such as the Rivest-Shamir-Adelman (RSA) 

public key algorithm or the data encryption standard private 
key algorithm. 

For those embodiments and materials data that download 
executable code segments, the implementation language 

2 0 importantly should address the security exposures thereby 
created. For example, a malevolent and knowledgeable user 
can create or modify the downloaded code to perform 
illegitimate operations within the client system or access 
restricted information from the server. The Java™ language 

2 5 is preferable in these embodiment because it now 

significantly addresses these problems and further 
improvements are constantly being made. Examples of Java"* 
security measures include limiting access to client system 
resources, particularly the file system and network 

3 0 connections, preventing downloaded software from "spoofing" 

local software, and providing byte-code verification to test 
code for possible security violations. Any implementation 
language for an ABI system preferably offers similar or 
improved security features. 
35 Additionally, it is advantageous to protect a portable 

student data object from being altered in an unauthorized 
manner, corrupted, or damaged. When AB2 system compliant 
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materials can be downloaded from multiple ABI systems or 
educational providers, particular care is advantageous to 
ensure correct functioning of these materials. Such student 
data object protection is accomplished by validating 
5 downloaded software and by encryption and key management for 
the student data object. 

Software validation includes registration procedures and 
runtime access control procedures. Before software or 
materials can be downloaded, they are preferably registered 

10 with an ABI system registry function, which can reside on a 
network accessible ABI server system. Before registration, 
proposed software is formally or informally inspected to 
ensure that it is safe, that is that it does not cause 
illegal or undefined system states. As previously described, 

15 a safety inspection is facilitated if the software is written 
in Java, which permits formal safeness checking. Also, the 
software is inspected to ensure that all modifications of the 
student data object are appropriate, that is that all 
modification reflect correct use of queried or updated 

2 0 fields. Upon passing these inspections, software is given a 

system identifier and its executable or interpretable forms 
are given a digital stamp by a checksum or cryptographic hash 
function to insure its identity. The system identifier is 
stored in the registry along with the digital stamp. 
25 At runtime, when the student requests access to 

particular ABI system compliant material or software from a 
provider, the student data object access control methods on 
the client system perform the following checking process. 
First, these procedures generate a request to the registry 

3 0 function to verify the software to be downloaded by providing 

the registry with the software identification. if the server 
responds that this software is not registered or authorized, 
the download request is rejected. If this identifier is 
validly registered, the registry returns the digital stamp 
35 associated with this identifier. The access control then 
downloads the software or material, recomputes the 
cryptographic hash function or checksum, and compares it with 
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the registered value. if they do not correspond, the 
download request is also rejected. if they correspond, the 
software is assumed to be safe and correctly identified, and 
the access control sets indicated access authorizations 
5 Such access authorization can limit this software to access 
only particular components of the student data object and 
with only read or read/update permissions. 

Additionally, during access to the student data object 
the access methods can consult rules so that clearly 
10 incorrect patterns of access can be rejected. For example a 
clearly excessive rate of parameter update can be caught and 
rejected. 

In addition to software validation, procedures for 
access control, encryption, and key management of portable 
15 student data objects are advantageous. As the portable 

student object is an important repository of agent knowledge 
temporary instruction materials state data, and, optionally, 
educational progress data, it is important that it not be 
altered or corrupted accidentally or maliciously. To achieve 
20 reasonable and effective protection, first, the portable 
student data object advantageously includes the student's 
password, or other identification means, in a protected form. 
When this object is plugged-in to a client system, a 
preliminary access control method insures that the student 
25 has supplied correct identification to the client system. 

Second, the student data object data is advantageously 
encrypted whenever it's data is present in a manner freely 
accessible by the client system. Therefore, in embodiments 
such as portable hard or floppy disks, the student data 
30 object data is stored encrypted on the disk. Portions are 
decrypted in memory only in response to specific access 
methods. in embodiments, such as smart cards or PCMCIA cards 
with processing, which provide for method processing as well 
as data storage, the data can be internally stored in plain 
35 text. However, when externalized, the data is encrypted. 

Such externa lizat ion is necessary in case student data object 
backup facilities are provided. For backup, internal data is 
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externalized as a encrypted, sequential, stream which can be 
stored and recovered from a backup system. 

Such student data object encryption can be supported by 
key management schemes known in the art. For example, a 
5 public-private key scheme can be utilized, A student's 
public key can be stored on the ABI system which processes 
student system logon requests, and returned to the client 
upon student logon. The private key is stored in a protected 
manner in the portable student data object. Thereby, after 
10 student logon to the ABI system and validation by the 

portable data object, both keys are available to decrypt or 
encrypt data as required. 

15 5.2.5. ABI System Dsaae Metering 

Each student receiving ABI instruction on a specific 
topic is presented with a combination of instruction, 
utilities, and on-screen agent interaction. Because the ABI 
System provides individualized interactions, each such 

20 presentation comprises a unique mixture of visual, audio, and 
software elements. Also, each user's consumption of hardware 
and communications resources differs. The ABI System 
therefore preferably incorporates metering technology to 
track the utilization of the different elements of the ABI 

25 System, Such information can be used to determine payments 
to the creators or providers of these elements and/ or as a 
basis of charges to the user. Further, this information can 
be used by ABI System providers to determine absolute and 
relative usage of various components of the system. Parties 

30 interested in such information include providers of server 
hardware, application software, networks, instructional and 
tool content, authoring tools and on-screen agent animations, 
scripts and utterances. Finally, this infonoation can be 
used to monitor system performance and utilization. 

35 Current metering technology does not provide the 

detailed measure of usage that is advantageous to ABI System 
providers. Such technology adapted for network distribution 
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Of content is available from, e.g.. the IBM Corporation, as 
the IBM Cryptolope, and Electronic Publishing Resources' 
(EPR) , as the Digibox. 

The ABI system comprises hardware elements, standard 
S software elements, special application software elements, and 
content elements typically supplied by different providers. 
Hardware elements include networks and other communications 
links, server CPUs, and disks. Standard software elements 
include operating systems, database packages, and 

10 communications software. Application software includes 
instructional playback engines, schedulers, and agent 
software. Content includes instructional materials, 
utilities such as dictionaries and encyclopedias, and on- 
screen agent animation and utterances. 

15 ABI metering comprises the elements of the metering 

utility software, of tagging of all ABI component elements, 
and of the preferred methods of monitoring. The pricing and 
compensation model is preferably provided by system 
administrators. The ABI Metering Utility Software ("The 

2 0 MUS") measures usage and maintains usage data in a 

'multidimensional usage model' during each student session. 
The MUS is resident at run-time on the student clients and 
the system server (s). In the student client, the MUS is a 
component function of the Session Manager with inputs derived 

2 5 from the Executive Software, Agent Software and the Materials 

Engine. In the Server system, MUS is a component function of 
the System Manager software. 

To facilitate metering, each ABI element is tagged with 
an ownership identification code similar to a product 

3 0 barcode. The first digits indicate ownership and subsequent 

digits indicate the specific component. As the elements move 
through the system from server to client and within the 
client, the use of these elements is metered. in the client 
software, the Materials Engine uses materials ownership tags 
35 to meter content. The agent utterance generation and visual 
display generation uses tags of the data snips to track 
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ownership of on-screen agent actions. The session and screen 
manager keeps track of which element of the screen is active. 

This invention is adaptable to any method for measuring 
usage appropriate to the particular component elements. in 
5 particular, four preferred methods of measuring usage are by- 
time, by-interaction, by-unit and by-packets. The by-packet 
method is performed at each server to track usage of 
communications resources. This method is preferably 
implemented by using a count of the number of bytes 
10 transferred in order to estimate the number of packets 
transferred. The ABI application software keeps track of 
counting the number of bytes transmitted to the user 
according to content type (i.e. system packets, instructional 
materials, or elf animations). Then based on the network 
15 transfer protocol used, the byte count may be converted to a 
roughly equivalent packet count. Alternatively, the byte 
count may be used directly as a measure. In the case of 
multiple servers, each server sends a summary of packet 
transmission information to a unique metering utility server. 
2 0 The by-elapsed-time method is performed at the client. 

In this method, the Screen Manager at the student client 
keeps track of the multiple simultaneous on-screen activities 
such as instructional materials, the elf, and the dictionary. 
This method preferably further monitors the active element of 
2 5 presentation, that is, the element performing an action or 
the focus for student input. In a more limited alternate 
implementation, the connect time of the student may simply be 
measured at the server. To avoid network problems such as 
lost connections and missed session signoffs, the System 
30 Manager application at the server additionally periodically 
obtains session statistics during a user session, either by 
polling the student client or from timed client downloads. 

The by-units method is preferably implemented either at 
the server or at the client. This method measures the 
35 content accessed by the client. For example in the case of 
instructional materials, the server records the speed *-'ic 
lesson accessed while the client records the comple on of 
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exercises. in the case of on-screen agent actions, this 
method records each completed display or animation sequence 
or utterance. 

The by-interaction method is preferably performed at the 
S client and records both fine-grained interactions, such as 
clicks and keystrokes, and the medium grained interactions, 
such as completed responses and meta-requests . This method 
when combined with the by-time or by-units methods provides 
an indication of the relative level of interactivity of the 

10 instruction. For example, during use of instructional 
materials, the by-interaction methods measures the actual 
usage of each resource such as agent processing. 

Usage of the different ABI elements is preferably 
metered by an appropriate method. The methods described 

15 above are not uniformly advantageous to different types of 
presentation. A graphics designer might seek compensation 
based on the amount of time a design is on the screen, while 
the instructional designer might seek compensation based on 
the fraction of the lesson completed. Accordingly, the by- 

2 0 elapsed-time method is preferred for graphics designers, 

while the by-units methods is preferably used to record the 
publisher and lesson for instructional materials, the type 
and author for tools material, and the creators of the 
animation and utterance for on-screen agent actions. The by- 
25 packet method is preferably used to record the source of the 
network traffic such as access to the student object database 
or downloading of application software, instructional 
materials, or animation data snips. The preferred metering 
methods provide information needed for the typical owner 

3 0 compensation and user pricing models. 

In its preferred embodiment, system administrators 
provide pricing and compensation models. This invention is 
adapcable to many alternative such models. One user pricing 
model is a flat rate based on the type of instruction being 
35 made available, similar to pricing in the cable industry. 
Another is based on connect time, as frequently done for 
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Internet access, or on type and quantity of information 
retrieved, as is common for on-line database access. 

TABLE lA illustrates an exemplary model for compensating 
providers of ABI system elements. Compensation is based on 
5 metering data and an allocation model. The four metering 
methods are applied selectively to different providers. The 
allocation amounts 'a', 'b', 'C and 'd' are externally ' 
determined. 

10 TABLE lA: TYPICAL MODEL OR USZKG METEKINO IK70RKATI0N 



25 



PROVIDERS 


COMPENSATION FOR PROVIDER 


Billing/ Network 


'a' percent 


Server Hardware Providers 


'bl' percent divided up according 
to number of packets with up to 
'b2' percent additional if total 
number of packets exceed a 
predetermined leve 1 


Instructional Materials 
Providers 


'c' percent divided up according 
to units 


Calendar/ Schedular Provider 


Up to 'd' percent based on level 
of interaction 


Animation Providers 


'e' percent divided up according 
to 'time on screen' 


ABI Client/server Software 
Provider 


Balance of revenue 







5.3. The iBStructional Intergace 

The ABI system has interfaces for students, teachers, 
30 and administrative staff. Materials and software developers 
can have specific ABI system interfaces. Alternatively, such 
development can occur on separate systems followed by 
indexing and loading of the developed components into the 
appropriate databases* 
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5.3.1. studeDt -fly«»*»«" Inf rface 

This section describes the structure and design of the 
student-system interface. This description is directed to 
the currently preferred implementation of this interface by a 
5 series of visual display screens. However, this invention is 
not so limited, and adequate alternative technologies, for 
example voice output with speech-recognition input, can be 
used to implement this design. 

10 5.3.1.1. Bx—plarv studOTt Scre#n3 

During materials presentation, a student sees screens of 
one or more screen types, all formatted in accordance with 
the previously described design principles. Screens are 
structured as a hierarchy of areas and subareas with text, 

15 graphical, animation, and perhaps video objects displayed in 
the subareas. Objects can either be for display only or 
permit student input or interaction. 

An important screen type is a task screen. Task screens 
are used for materials presentation tasks such as homework 

20 assignments including problems, programmed steps, mastery 

quizzes, and drills. Table 1 and Fig. 3 illustrate exemplary 
components of a task screen appropriate for elementary 
education. Reference numbers in Table 1 are from Fig. 3. In 
general, in system area 302, the session manager presents 

25 objects visualized as icons permitting the student to easily 
access ABI system facilities. In materials area 304, object 
presentation including description, placement and movement, 
is specified by the materials designer in the presentational 
and sequencing sections of the materials data. The materials 

30 engine interprets these specification at run time to send 
display objects for display. In agent area 303, agent 
behavior processing formats predefined parametrizable 
objects, including resolution of object parameters, 
representing selected agent personae. The output from all 

3 5 selected and formatted display objects is sent to the 

executive software and operating system for ultimate display 
to a student. 

- 52 - 



_BNSDOCID: <WO 9744766A1_I_> 



wo 97/44766 



REF 
NO. 

301 
5 302 
305 

306 

309 
307 
308 



10 



20 



25 



30 



35 



310 



304 
15 311 
312 
313 
314 
315 



316 



321 
317 

309 



303 



318 
319 



320 



PCT/US97/08685 



TABLE 1: STX7DEKT SCREEN DISPI«AY ELEMENTS 



NAME OF ELEMENT 

Screen 
— System Area 

-Customizable 



— File System 

— Scheduling Toolbar 
I — Monthly Calendar 
j — Clock 

— Toolbar 



Materials Area 
— Open Book 

— Section Tabs 
— Exercise Area 
— Header 
— Presentation 



--Input 



— Navigation Buttons 
— Student Action 



— Sched • /calendar 



On-screen Agent Area 
— Audio 

— Text Message 

— Visual Persona (e) 



— Agent Recfuest 



DISPLAY OUTPUT OBJECTS 

Entire ABI display area 

Graphics and text 

(can be animated) 
Book icons with titles 



Graphics 

Dynamic graphics 

Icons of tools (for 
calculator, word 
processor , dictionary , 
communications , etc . ) 



Graphics of book outline 
Graphics with text 
Text, graphics 
Text, graphics 
Text, graphics, 

animation, video 
Interactive objects 

(buttons , etc. ) , 

filling in text, 

dragging, drawing 

Graphics 

Icons for Submit, Help 

Graphics for calendar 
with text of student 
schedule, icons for 
scheduling options 

Synthesis of spoken 
utterance 

Text utterance 

Appearance integrated 
with audio, graphics 
background effects 

Interaction 

with other screen areas 

Agent meta-request icon, 
generates pull -down 
menu of meta-requests 



In more detail, the task screen of Fig. 3 includes in 
system area 3 02 student customization area 305. In the 
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customization area, the student can display objects given by 
the agent as rewards for student accoraplishment . These 
objects can include text, as illustrated here, graphics, or 
animations. Another part of the system area is file system 
5 toolbar 306 displaying accessible files as icons in a "book- 
on-shelf" metaphor. This invention is adaptable to other 
reasonable file system display metaphors. Here, for example, 
the book icons represent a file personal to the student, a 
file holding ongoing instructional materials, a file of e- 

10 mail, and files for tools such as a dictionary and group 
activity. Below file system toolbar 306 is toolbar 310 for 
tools the student has access to. Illustrated here are icons 
for a calculator, a word processor, communications, and 
starfish, a general purpose language tool. 

15 "Starfish" are a visualization tool for semantic 

networks that can be available in an ABI System. a semantic 
network typically consists of nodes linked to other nodes by 
semantically significant and named links. A starfish tool 
visualizes such a network for a student by displaying the 

20 node centrally in the body of the starfish with the node's 
links displayed as arms of the starfish. For example, a 
dictionary represented as a semantic network might include a 
node for each word with links to similar words, opposite 
words, root words, and so forth. A dictionary starfish 

25 displays a word in its body. Each selectable arm is labelled 
with a category - synonym, antonym, pronunciation and so 
forth. Clicking on an arm causes the corresponding word(s) 
to be displayed or spoken. Such a tool provides integrated 
display of diverse knowledge structures for a student. 

30 The scheduling/calendar tool is an important tool that 

is always available. Exemplary icon 309 illustrated for this 
tool has calendar part 307 and clock part 308. Selection of 
each of these parts brings up daily and monthly scheduling 
functions. These function can either prescribe the student's 

35 next activity or permit choice where the student has excess 
time or demonstrated personal scheduling ability. 
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In materials area 304, instructional materials, tools, 
and communications materials display their content. 
Illustrated in Fig. 3 is page 3 of an exemplary mathematics 
homework. Instructional materials are advantageously 
5 structured as a book of exercises and items, emulating 
current textbook and workbook practice. In this case, 
section tabs 312 permit the student to navigate the homework 
book by sections, and page buttons 321 permit the student to 
navigate the homework book by page. Also present is toolbar 

10 317 of available student actions, which for homework includes 
submit and can, if the agent permits, include help and hint 
requests. The format of a materials page is advantageously 
standardized. An exemplary standardization has header 
information 314, presentation 315, and interactive input area 

15 316. 

On-screen Agent area 3 03 allows the student entry of 
raeta-reguests and allows the agent to display synchronous or 
asynchronous meta-responses . This exemplary area illustrated 
in Fig. 3 comprises meta-recjuest button 320 which the student 

20 can activate to display a list of currently available meta- 
requests. Student meta-requests are generally allowed by the 
ABI system at any time. Requests may occur at the beginning, 
middle or end of instructional sequences, in choice screens, 
in tools, in group communications, or in the calendar. Agent 

25 meta-responses are individualized and context dependent. 
Typical meta-requests are illustrated in Table IB. As an 
example of context dependence, if the request 'Where am I?' 
occurs within a lesson, a graphic representation of the 
lesson will be shown; whereas if the same request is made 

30 between lessons, a graphical representation of previous 

lessons and new lessons will be shown. The agent software 
attempts to respond based on the user's intention and uses 
the context and previous requests to infer the reason for the 
current request. 

35 
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TABLE IB: TYPICAL MBTA-RBQUESTS 



10 



TYPE 


META-REQUEST 


TeniDoiral ! Time rema i n i nrr i n 

current: unit/session/days work 


'How far along?' 'How much 
longer? ' 


Markers nan^ r^oor^fi n 
current lesson/exercise; relative 
position in lesson 


Where am I? ' 


Pause: Repeat of instructions: 
summary of what has been done 


'Say it again,' 'Say that 
another way . ' 


Performance: summary of work in 
current lesson 


'How am I doing?' 


Choice: Branch to take, e.g, 
which lesson to do next 


'What do I do next?' 


Purpose: function of current 
lesson 


'What is this for?' 


Resources: assistance available 
at this point 


'I need to hint.' 'i need 
help. ' 



The instructional materials notations provide a 
framework for the agent software to keep track of the 

20 relative position of the student in the lesson. This same 
information is also available to the teacher for lesson 
preview and annotation. 

The remainder of area 3 03 is for agent meta-responses , 
which importantly have multi-media structured into personae. 

25 Illustrated are text message 318 and visual persona 319 that 
typically includes animation. Also possible is audio output, 
either text-to-speech or generated from audio files. 

Fig. 4 further illustrates an exemplary screen 
interaction between the materials and the agent. Fig. 4 

30 shows only the content of materials area 5 01 and on-screen 
agent area 502 of the complete display screen of Fig. 3. a 
mathematics homework material is displaying item presentation 
503 with input selection buttons. The student has selected 
wrong input button 504. At this educationally significant 

35 event, the materials send to the agent several messages 

generated by notations in the materials data. In response, 
the student's agent has chosen to act ajs illustrated. First, 
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it displays text 506 of the rule violated by the student 
answer. This text was sent to the agent by the materials in 
an event message for its use. Second, the on-screen agent 
points 505 to the screen location of the error. This 
5 location was also sent to the agent by the materials. Third, 
perhaps in response to a previous high or increasing error 
rate of the student, the on-screen agent presents a meta- 
response 508 commenting on the pedagogic nature of the 
student's error. Further, it activates a persona 507 to 
10 engage the student's attention. This persona can 

advantageously include animation, audio, and speech output of 
the displayed text. Thus, the agent software integrates 
speech utterances, visualization, display of text and 
graphics, and animation into a persona display for 

15 highlighting an educational event that the agent determined 
important based its processing of the current input, past 
student inputs in this lesson, and the student's pedagogic 
model generated over several sessions. 

Other screen types are of importance in an ABI system. 

20 For example, tools can have distinctive screen types 

appropriate to their nature, such as a calculator image. 
Choice screens can be used at the transitions between 
instructional sequences. They summarize what instructional 
materials have just been completed, announce status 

25 information, and list any materials choices now available to 
the student. These choices can related to instructional 
materials to be undertaken next or to optional, non- 
instructional materials, such as exchanging credits for time 
in a game room, access to e-mail, and so forth, that can be 

30 available to the student, 

5.3,1*2. atudant Actions 

The inputs of a student or other user of the ABI System 
are preferably classified as requests, meta-requests , or 
35 data. A student request is an input directed to materials or 
to the system seeking a specific action. Student data is an 
input responding to a system request for information. For 
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example, student requests include an input to the system to 
start the calculator is a request, or an input to certain 
materials to submit completed homework to the teacher. 
However, input of numbers into the calculator is data. This 
S invention is adaptable to a further particular type of data 
input, termed semiotic, in which the student selects certain 
signs or symbols to provide input. Semiotic input is 
particularly advantageous when the agent requests the student 
to report his feeling or appreciation for a particular 
10 educational event. Meta-requests are inputs directed to the 
agent seeking specific actions. For example, student input 
to the agent seeking hints during the current materials is a 
meta-request . 

Displays produced by this invention are preferably 
15 classified as applications, responses, meta-responses , or 

questions. Application display occurs upon system initiation 
of an available material, for example, an instructional 
material or a tool. Application display usually occurs in 
response to a student request, but can also occur on system 
20 initiative. For example, the agent can request the system to 
make a tool available to the student. Responses are all 
displays produced by materials or by the system. For 
example, responses include materials presentation display and 
display on the adequacy or correctness of student input. 
25 Meta-responses are all displays produced by the agent. These 
can be synchronous, generated by responses to a student meta- 
request, or asynchronously, generated by events sent to the 
agent from the materials. Questions are a particular form of 
response or meta-response which seek further student input. 
30 Questions engage the student in a form of a dialogue which is 
an integral component of the tutoring interaction. 

Fig. 5 illustrates an exemplary gegment of the 
interaction of the student and the system that can occur 
during a mathematics homework. It illustrates both how 
35 responses and requests are distinguished from meta-responses, 
and meta-requests and also how the agent, through its 
observation of the student's current situation and its 
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contact with past student history, is able to guide the 
student better than the materials alone, which are only aware 
of the current context. This display is individualized to 
the student's current and past performance and preferably 
5 uses realistic, life-like on-screen personae to engage the 
student. Displays from the ABI System directed to the 
student are indicated generally on the right; inputs from the 
student directed to the ABI system are indicated generally on 
the left; and time increases downward. The diagonal arrows, 

10 as at 4 01, indicate that the message at the arrow's tail 
leads to the message at the arrow's head. 

This illustration begins with response 4 02 from the 
mathematics homework materials suggesting that the student 
use the calculator tool. The student responds with request 

15 4 03 which selects the calculator icon to request the 

calculator. Calculator application 404 is then activated and 
displayed by system components. Data 4 05 is a student input 
to which the calculator tool, and response 406 indicates a 
student error. This response was apparently not sufficiently 

20 informative, and data 4 07 is a repeated incorrect input to 
the calculator tool. The student's agent observes the course 
of this interaction as the calculator tool sends event 
messages to the agent. Upon observing the repeated wrong 
input, the agent intervenes with a asynchronous meta-respbnse 

2 5 4 08 summarizing recent student inputs. The student responds 

with meta-request 409 to the on-screen agent for help, since 
the tool responses were apparently not sufficiently helpful. 
The agent's synchronous meta-response 410 suggests for 
consideration an entirely different approach to correct the 

3 0 student's recent error. 

5.3.I.3. Bvsta» Responaas 

The following general principles preferably guide system 
and materials responses and agent meta-responses. First, 
35 some display is to be made on every user input, if only to 
echo a character or mouse click. Second, a user is never to 
be left in doubt about the status of current tasks. For 
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example, the system preferably provides task specific hints 
or suggestions if no user input is received in a time period 
adaptively determined^ Third, all responses reflect the 
current context* 

5 Further, importantly, aspects of system responses should 

be adapted to the particular student audience, from 
elementary to adult education and including special classes 
of students. This tailoring can be set by the teaching or 
administrative staff. One adaptable aspect is the language 

xo level and the language of system responses - for example, the 
vocabulary and language of help services, messages, and 
tutorials - is preferably adjustable. Another important 
adaptable aspect is the type of personae of the on-screen 
agent is preferably adjustable. The types of encouragement, 

15 level of jokes, and so forth, depend closely on the intended 
student population and are advantageously adjustable • 
Further, tools are also adaptable. Each tool is also 
advantageously enabled or disabled for each student. Mail 
and communications tools, for example, can be made 

2 0 unavailable for the lower grades. File creation and deletion 
can require strict controls- Also, certain tools can have 
differing complexity levels suitable for differing student 
levels . 

In addition, certain ABI system components have 

2 5 particular preferable and advantageous features. These are 

listed subsequently. Importantly, meta-response selection is 
preferably shaped in view of the student's past baselines of 
speed, performance accuracy, modality, as specific to the 
type of materials. Moving average functions, in which recent 

3 0 values are given higher weight than earlier values, can be 

used advantageously to generate baselines of performance and 
timing. The timing of meta-responses is preferably based on 
student data object fields that record the amount of help 
asked for in the past and past performance provided. 
3 5 Further, the pacing of meta-responses is advantageously 

context sensitive. For example, remediation should be 
offered only in case of repeated error or hints offered 
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asynchronously only after a relatively long period without an 
answer. Also, previous student interactions should be 
utilized. Repeated requests for hints should be noted and 
dealt with perhaps not with another hint but with 
5 remediation. The rate of prompts, advice, and hints should 
be adaptively adjusted on the basis of ongoing performance 
records. 

The actual content of a meta-response can be adjusted to 
the current situation by filling in parameters from event 

10 messages sent from the materials. See Infra. The 

Instructional Materials Interface Standard ("IMIS") is a key 
element in providing information to the student in response 
to inquiries such as "Where am I?". The system then uses the 
information in the IMIS to respond, "You are halfway through 

15 the drill and practice on fractions," for example. A student 
request for further information can elicit a graphical 
response. The graphical responses reinforce the virtual * 
tutoring aspect of the system by providing a greater level of 
detail in a format similar to that presented by a human 

20 tutor. The format of the graphical method used is 

individualized to the student's past use of system components 
such as previous graphical meta-responses and tools such as 
the scheduler/calendar . 



25 construct graphical responses is derived from lesson 

notations, previous student performance, previous agent 
responses (including graphical responses) and the 
scheduler / ca lendar . Marker requests use general notations 
for the type of lessons and specific notations for 

3 0 prerequisites. Temporal requests use notations of expected 
time together with profile information on individual 
performance on that type of material. 

Typical predefined graphical response templates include 
road maps (perhaps in the form of a train and subway map) , 

35 charts, table, graphics and icon flow charts. Colors can be 
used to the show relative levels of performance, urgency or 
dependency of elements. In response to a 'How am I doing' 



The structural information for the agent software to 
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request^ green might indicate good perf onnance ; yellow, 
adequate performance; and red, needs immediate attention. 
Similarly, the shape of the icon might indicate its function 
such as a diamond for caution or a hexagon for stop. 
5 Figure 12 illustrates an exemplary dialogue starting 

with the student meta-request 'where am I?'. The on-screen 
agent then make a meta-response containing a summary of the 
student's progress, for example "You are one quarter through 
the lesson on adding fractions." Next, the student might 

10 request further information. Preferably the on-screen agent 
will point to the location on the diagram and say "You are 
here." If the student then clicks on an element of the 
chart, the agent will provide further information • tor 
example, clicking on the Practice Box will provide a meta- 

15 response containing a flow chart of this upcoming activity, 
as generally illustrated. 

Preferable Calendar/Scheduler Responses And Their SeXection 

The calendar scheduler is advantageously capable of 
20 providing the following responses. These are controlled by 
schedule data fetched from school-student data areas and are 
processed by the calendar/ schedule tool. Preferably, the 
student data object contains only personal schedule data for 
the student. The calendar/schedule tool and the student's 
2 5 agent can obtain schedule data both from school-student areas 
and from the student's personal schedule areas. 

1. Schedule reminder responses, which remind the 
student of deadlines for materials tasks in the system or for 
external activities, such as getting a parent's approval for 

30 a class outing. 

2. Task sequencing suggestion responses, which suggest 
an order of assigned tasks based on student history and on 
the assigned priority and deadline* 

3. Timing estimate responses, which estimate how long a 
35 task will take based on timing information entered as part of 

task in the instructional materials and on past relative 
performance for this student. 
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In one embodiment, the ABI system provides for of 
scheduling initiative to be divided between the student, and 
the system. Settable task schedule parameters permit, at one 
extreme, the student to have complete scheduling control, 
5 able to initiate and to exit any activity at will, and limit, 
at the other extreme, the student to work on only those 
materials that the system schedules. These schedule 
parameters include those controlling the tools and options 
available to the student while performing a given task and 

10 those requiring the student to perform background reading or 
remediation. Important initiative parameters include the 
scheduling values of task priority value and deadline. If 
the priority and date are not both "off", those tasks with 
greater priority and earlier deadline are automatically 

15 scheduled for the student. If these values are "off", the 
student has control of task scheduling. 

The system of this invention also includes a central 
list of timed activities, perhaps stored on the server 
systems, to be performed by the system. These can include 

20 regular times for the generation and printing of standard 
reports or for the broadcasting messages concerning group, 
activities such as spelling bees. These timed activities can 
be performed by scheduling software which compiles the timed 
activity lists and initiates the listed activities at their 

25 listed times. Such scheduling software can be similar t:o the 
"at" utility in UNIX operating systems, 

S»3*2« Teacher /^*<»<7* istrator-flv3teM Interface 

In the case where a teacher supervises a student's 
30 education, the teacher or the teacher's representative plays 
an important role in the ABI system. The teacher uses the 
system to perform such functions as entering initial profiles 
in student data objects and school -student data areas, 
assigning students to subgroups, previewing, annotating and 
35 scheduling assignments, reviewing and commenting on completed 
homework assignments, and reviewing summary reports. 
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In this setting, the agent of the student is also an 
agent of the student's teacher in that the student's teacher 
controls key parameters in the student data object and the 
school-student data area, which in turn can direct agent 
5 actions. The teacher customizes the ABI system by 

initializing and supervising school -student area and student 
data object parameters, assigning and prioritizing 
assignments, and customizing materials. Important teacher 
activities are included in the following list- 

^0 3., The teacher initializes and exercises continuing 

control over important data in the school -student data, and 
in this manner supervises the student's use of the system. 
For example, the teacher controls the access and level of 
tools available to the student and limits the extent to which 

15 the student can alter agent personae. 

2* The teacher controls the student's use of the ABI 
system by assigning, scheduling, and prioritizing the 
student's access to the materials. This is accomplished by 
teacher control over the schedule subtype in the school- 

2 0 student data areas. For example, the teacher can schedule 

tasks that must be completed on the ABI system, schedule non- 
system tasks, remove tasks or modify their priorities, 

3. The teacher can customize materials available to the 
students. The extent of routine customization includes 

25 modifying sequencing of instructional lessons, elements, and 
items, choosing the homeworks the student must complete, 
specifying the formats of homework assignments having some 
student discretion, such as reports, sending messages to 
students . 

30 4, The teacher's class management is aided by a 

facility to send messages, reminders, hints, etc. to students 
using the ABI system e-mail facilities. 

The system can advantageously assist the teacher in 
homework management. Once the student completes and submits 

35 a homework assignment, a printed copy can be made for the 
teacher and the student. The homework assignment can be 
graded by the ABI system, if answers were provided as part of 
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homework material. The teacher can add comments for the 
student, if homework is viewed online by teacher. 

The system can advantageously also provide the teacher 
with sumjnary and detail reports for each student and class. 
5 These reports can be immediately available online or printed 
for later review. As known in the art, reports can contain 
both current and cumulative data on instructional progress 
and homework assignments* The reports can also flag patterns 
of deficiency in a student's homework and problems in a 

10 student's instructional progress. In a preferred embodiment, 
these reports are generated from the database of school- 
student data forwarded to the school server systems by the 
students' agents. 

Additionally, the teacher can be a student. A teacher 

15 can benefit from training in the use of the ABI system in 

general, in the procedures to customize materials, and in the 
characteristics of the particular materials used in that 
teacher's class. This training can advantageously be 
packaged as instructional materials directed to the teacher 

20 which are otherwise similar to student instructional 

materials. In this case, the teacher, like the student, has 
accessible materials, a teacher data object recording the 
teacher's progress and pedagogical characteristics, and an 
agent using this data object to guide the teacher's training 

25 and guide the teacher in the use of the system. As for the 
students, the teacher data objects can advantageously be 
stored as portable teacher data objects. 

Other actors, such as school administrative staff, 
parents, and researchers, can play a role in the ABI system. 

30 Administrative staff can have privileged access to certain 
data items in the student and teacher data objects and other 
system data, which permits them to assign students to 
courses, to assign students to teachers, and to establish 
instructional performance standards and criteria which the 

35 students must meet to complete their materials • This staff 
can also receive online or paper reports on the progress of 
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Students in the schools, the effectiveness of teachers, and 
the usefulness of the particular materials assigned. 

Reports and queries 

5 Generation of reports from databases, either relational 

or object-oriented, is a standard programming task. The key 
elements of this task are the selection of data to appear in 
the report, the way in which selected data is to be presented 
whether in a summary form, a detail form, or both, the format 

10 in which the data is output, and the layout of data output 
elements on the screen or page. 

The teacher and administrators require several types of 
reports. Periodic reports need to be generated at regular 
intervals. These vary according to population: by student, 

15 by class, by grade, by school. They also vary by subject. 
The reports can present/ summarize complete sets of data or 
only data exceeding stated limits. Some of these reports 
fulfill mandatory reporting and record keeping requirements, 
and some are sent to parents. The ABI system also provides 

2 0 teachers and administrators with standard querying and report 
generating capabilities. 

If appropriate, parents can also be actors in an 
embodiment of the ABI system. A student's parents can be 
given access to certain fields in their student's data object 

2 5 and school-student data areas in order that they can receive 
rapid information on their child's assignments and 
performance. This information can be made available at home 
on the same client system that their student receives 
instruction and homework. 

30 Finally, educational researchers can receive certain 

access to ABI systems in order to research the effectiveness 
of educational theories and methods. For example, they can 
efficiently compare the effectiveness of various educational 
paradigms in certain instructional contexts by receiving 

35 reports relating to students pursuing materials constructed 
according to the paradigms of interest. 
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5.3.3. 



Instructional oesignar-systea Interface 
Dasiqner Interface Overview 



Materials, in particular instructional materials, are 
authored by instructional designers. Authoring of materials 
g can be done on the system on which the materials are to be 
used, or alternatively, on a separate system. Generally, an 
instructional designer authors materials including, for 
example, computer assisted instruction as known in the art, 
computer assisted exercises such a homework or simulation, 
10 computer managed student instructional tasks which can 

involve work with several materials. For all materials, the 
student's agent must be informed of the sections completed 
and skills acquired in standard formats. 

Designer Interface Details 

The ABI system provides an environment in which the 
student's agent is available to control materials 
presentation and guide the student to improve educational 
outcomes. This environment includes facilities to present 
assignments, assess responses, probe for prerequisites, offer 
assistance with tools such as dictionaries, and score unit 
mastery quizzes. The system can advantageously include 
conventional forms of homework, such as work sheets, as well 
as new types of homework, such as group-based homework, and 

25 formats previously too time consuming for teachers, such as 
criterion based rather than fixed length or interactive 
rather than prescriptive. Advantageously, the system 
structure can accommodate existing forms of computer assisted 
instruction by embedding such existing instruction in 

30 ™^^^rials of this invention which contain notations and 
generate agent event messages. 

All authoring information below can be provided by the 
instructional designer, if preferrel. In this case, defaults 
allow the teacher to enter only overriding information where 

^5 needed. Alternatively, the teacher can be allowed or 

required to make various degrees of customization. In this 
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section, the term "exercise" means a single unit of an 
instructional task. 

The run-time conversion of materials with descriptions 
of text, graphics and animation advantageously is performed 
5 by methods commonly used in interactive multimedia Authoring 
Software • Existing authoring programs let the author specify 
the attributes and behavior of multiple independent display 
objects through such programming paradigms as visual 
programming or scripting languages. For example, to display 

XO a button or other selectable object, a user of authoring 
software specifies such characteristics as position, 
attributes such as size, shape, color, and label, and the 
function invoked by the object selection. 

Among the most widely used of the many available 

15 products are Macromedia's Authorware and Director, 

Asymetrix's Multimedia Toolbook and Aimtech's IconAuthor. 
These programs have been documented by their manufacturers 
and other writers (Sims, Authorware with Style; Swear ington 
et al., MaaromBdia Director Design Guide; Hill, Using 

20 Multimedia Toolbook 3.0). 

The process of authoring an instructional task is 
advantageously described sequentially to include the steps 
of, for example, defining the task, defining each exercise, 
judging student inputs, defining exercise completion, and 

2 5 augmenting the materials with notations for the agent. 

In the authoring step of defining the task, the 
instructional designer advantageously provides information 
including initial instructions for the entire task in a 
written and preferably also a spoken format with alternative 

30 wordings as necessary for special classes of students. The 
designer also provides materials sequencing in accord with 
the education paradigm chosen. Instructional sequencing is 
appropriate for interactive instruction with feedback. 
Homework sequencing can include a fixed order; a student 

35 defined order; a student defined order assisted by teacher 
prioritization into such priority groups as essential. 
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important, or optional; and sequencing defined by performance 
to criterion on individual item subtypes. 

In the authoring step of defining each exercise, the 
instructional designer advantageously provides information 
5 including the following. The designer chooses names for this 
task and exercise, its prerequisites, and the skills to be 
acquired in accordance with school system standards so that 
the agent can meaningfully track student performance and 
provide helps, hints, and remediation. Next, the designer 

10 provides student instructions for the exercise, preferably in 
both spoken and written formats. Next, the designer 
specifies the presentation of the exercise by creating 
graphics and text for the exercise and by specifying their 
display attributes. Next, the designer defines allowable 

15 user inputs, the data type of input, and the display areas in 
which input is allowed. Finally, the instructional designer 
provides hints. Hints preferably include optional hints 
offered on student request or student inactivity and hints 
offered by the agent in case of totally inappropriate 

20 actions. 

In the authoring step of judging student inputs, the 
instructional designer advantageously provides information 
including the following. First, the designer defines 
standards for the completeness of requested inputs and 

25 actions, possibly none, in case of incomplete inputs. An 

exemplary such action is suggesting the completion of omitted 
inputs. Second, the designer selects how inputs are to be 
judged and the possible judgements, for example, correct, 
near miss, incorrect, and so forth. Finally, the designer 

3 0 selects error responses to individual input elements or an 
entire exercise. These responses can be uniform or can 
reflect the specific wrong answer. Error responses include 
retry policies and selection of remediation paths. 
Remediation can be item sequencing options given on the basis 

35 of retry outcome. 

It is preferable that response judging be performed 
according to certain standards established for a particular 
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implementation of this invention. Thereby, the agent can 
monitor the student's pedagogic characteristics in a uniform 
fashion across all standardized materials. Accordingly, a 
first step in input judging can be, where appropriate, 
5 checking the data type and perhaps range of an input. For 
example, checking that a fraction is entered in response to a 
fraction addition problem. Appropriate feedback can be given 
if the wrong data type or range is input- Single student 
inputs can include entry of a single text or numeric element, 

10 movement of a single display object, selection of a single 
display object, or drawing a line or curve perhaps encircling 
a display object. 

A second step in input judging can be classification of 
a suitable input in standard error categories. In an 

15 exemplary embodiment, this can be performed by providing a 
pattern corresponding to a correct answer. The patterns can 
be a template appropriate to the suitable input, such as a 
text template or a spatial template. A pattern matching 
procedure selected from a system library of such procedures 

2 0 can be executed to match the input to the pattern. The 
output of the system matching procedure is advantageously 
selected from a limited number of standard judgements of 
correctness and partial correctness. This judgement is then 
communicated to the agent in an event message in order for it 

25 to guide its student and adapt its pedagogic model. This 
invention is adaptable to templates and pattern matching 
procedures known in the art. Patterns can include one or 
more parametrized text templates, spatial templates defining 
the relative positions of objects to be moved by the student, 

30 selection templates specifying patterns of selections. 

Further, spatial templates can include outlining areas of the 
screen, shapes of drawn objects, maze traversal, and tool 
inputs. 

Turning to the authoring step of defining task 
35 completion, the instructional designer advantageously 

provides information including standards for input, exercise, 
and task completion. The materials can require a certain 
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minimum completion before allowing homework submission to the 
teacher. 

The final authoring step is augmenting the materials 
with additional notations for the agent. These notations 
5 concern task and exercise subject, skill classifications, and 
definition of the educational paradigm embodied in the 
sequencing logic. It is performed by instructional designers 
or knowledgeable teachers, and is expressed as notations in 
the materials that generate event messages for the agent and 

10 that reference control parameters set by the agent for 

control of materials sequencing. The notations so entered 
communicate various types of information to the agent. 
Exemplary types include charac -erization of assignment by 
type of task, subject, elements of subject covered by task; 

15 comparison between exercises in same assignment by similarity 
and difficulty; the subject matter prerequisites to determine 
what the student may or may not have mastered based on which 
exercises are answered correctly; system knowledge 
prerequisites; and scheduling information such as average and 

20 expected time to complete. 

5,4. ABI SvatMi Software Structure 

This section describes in a more detailed fashion an 
exemplary structure for the software components previously 

25 described in a more general fashion with reference to Fig. 2. 
Subsequent sections describe particular components in an even 
more detailed fashion. The structure described here is 
exemplary. This invention is adaptable to other structures 
with other allocation of the functions of this invention to 

30 different modules. Such alternative structures are easily 
designed by those of skill in the arts. 

This section first describes the principal client 
software and data components and the conceptual hierarchy 
which reflects their interaction. Next, the structure of the 

3$ executive software is described. Finally, the flow and 
processing of events through the system are described. 
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Student Client Boftvara fltruetiuyo 

Fig. 6 illustrates the principal client software and 
data components and the conceptual hierarchy which reflects 
their interaction. The links illustrated represent principal 
5 procedure call or class dependencies among the components. 
At the root of the hierarchy is operating system 601, which 
has been previously discussed in connection with the 
preferred client hardware. The operating system ("OS") 
includes the operating system proper together with 

10 implementation languages and their class and function 

libraries with associated controls. At the next level is 
executive software 602 ("ES"). This software collects a 
number of components which customize the operating system to 
the requirements of this invention and also extend it, if 

15 needed. For example, all OS task creation is processed 
through an ES task control facility to insure that the 
student accesses only permitted materials. The ES software 
also provides a preferred animation facility and controls 
client startup. 

2 0 Session and screen manager 603 is always present on a 

client system. This component partitions the screen into the 
areas for the principal system components, as has been 
generally illustrated by the exemplary screen of Fig. 3, and 
controls the system area, area 3 02 in Fig. 3. Thus, upon 

25 student selection of an icon presented in the system area ' 
requesting the ES to start the function represented, it turn 
checks with schedule/ calendar 6 07 whether the student is 
currently permitted to access this function before continuing 
with OS task creation. The system manager also presents 

30 whatever reward graphics and animation the student has been 
granted access to. These functions are performed by calling 
the object level I/O facilities of the OS and ES. 

Also accessible on a client system is student data 
object 611 for the student in session on the client. This 

35 object, preferably including ABI system data relating to the 
agent's model of the student, to the student's current 
instructional state, and to the instructional materials 
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themselves, is preferably accessed as previously described 
from portable media. Alternatively, it can be downloaded 
from the server systems when the student logs onto the 
client. Preferably, this object is divided into components 
5 including methods and data subtypes, and those only those 
components required are actually accessed at any one time. 
As indicated by double headed arrow 616, the student data 
object is accessed and updated primarily by agent processing 
functions 609 and 612. 

10 In a preferred embodiment, school specific data is 

separately stored in school-student data areas 615. As 
indicated by* arrows 608 and 614, this data is preferably 
accessed by ABI client system components other than the 
agent. School-student data is preferably accessed from 

15 databases on school servers. In alternative embodiments, 
various subtypes, or all, of school-student data can be 
combined into student data object 611. 

Schedule/calendar component 607 is important for 
management of the student by other system actors, such as 

20 teachers and administrators. It is a tool that is always 
active whether or not it has a screen presence. Activities 
for the student, represented by activity description, 
priority, and deadline, are entered into the 
schedule/calendar data subtype of the school-student data 

25 areas by teachers or administrators. Optionally prerequisite 
information can also be entered into the schedule/calendar 
subtype. As indicated by arrow 608, this component 
references this schedule/calendar subtype data as well as the 
time expected for the student to complete an activity, as 

3 0 determined from the student's past performance and stored in 
the student data object, in order to determine whether 
certain activities, such as particular instructional 
materials, should be automatically started by the ES without 
any student intervention. In cases where the student is 

35 permitted greater initiative, the schedule/calendar can 
permit OS task creation requested by the student, as by 
selecting an icon. This component also provides, on request, 
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prioritized lists of tasks for the agent to present to its 
student when the student asks for advice on what to do next. 
It uses priority and deadline information and time-for- 
completion data computed using data in the instructional 
5 materials and student data object. 

The materials are represented in the ABI System by 
materials engine 604 and materials data 605- As previously 
described, the materials data comprises objects for display 
output or student input, sequencing logic controlling object 

10 display, and notations which when referenced by the 

sequencing logic during object presentation generate event 
messages for the agent. Materials data is advantageously 
grouped into "entries" comprising those objects, logic 
sections, and notations related to the display of a single 

15 screen or a few screens related to a single educational item. 
The materials data also preferably include a special header 
or initialization entry with important notations that 
communicate to the agent the educational paradigm adopted by 
these materials and can inform the agent of the control 

20 variables to which it is responsive. Such header or 

initialization messages are also preferably sent to the agent 
if the materials change their education paradigm. In a 
preferred embodiment, there is one common materials engine 
604 which presents a plurality of materials data 605. In 

25 alternative embodiment, there can be specialized materials 
engines for various classes or types of materials data. 

Materials engine 604 downloads the entries of materials 
data 605, interprets the sequencing logic, displays objects 
as requested, and references embedded notations generating 

30 the requested event messages to the agent. Standard 

facilities of the OS and ES are used for object presentation 
and for routing any input to the materials. Input is 
processed according to specifications in the sequencing 
logic. In a preferred embodiment, the agent controls the 

35 materials by setting shared global variables which the 

sequencing logic checks. Such shared global variables can be 
made available by OS inter-process communication ("ipc") 
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facilities, such as shared memory. The materials engine can 
be any program implementing these requirements, in particular 
an extension of available commercial authoring tools and 
engines, such as Macromedia's Authorware, 
5 In an alternative embodiment, the materials are not 

separated into an engine and a data component but consist of 
one unitary component for each material. In this embodiment, 
a process similar to compiling creates a single executable 
^aaterials component. This invention is equally adaptable to 
10 other implementations of the materials that generate agent 
messages and are responsive to agent control according to the 
requirements outlined herein. 

Agent processing is divided into two components, agent 
action processing 609, which determines agent display actions 
15 in response to events, and agent behavior processing 612, 

which transforms display actions into displays of personae to 
the student. As indicated by arrow 616, both components 
access student object 611. In a preferred embodiment, agent 
action processing is rule based and event driven. Rules in 
20 rules tables 610 are evaluated using parameters both 

communicated in event messages from the materials or the 
student and retrieved form the student data object. These 
rules propose candidate actions and then weigh and select a 
final set of agent actions from the candidates, which are 
25 communicated to subsequent agent behavior processing 612. 
Agent processing also sets global variables for materials 
sequencing and control. A side effect of this processing is 
the updating the student data object with information from 
the materials event message. 
30 Agent behavior processing 612 constructs an on-screen 

agent display based on the actions determined in agent action 
processing. In a preferred embodiment, this processing is 
based on behavior tables 613. Utterances, text or voice, and 
affects are selected from tables based on the determined 
35 final actions and refined with parameters included with the 
actions. The utterances and actions are sent to the selected 
agent persona object, which creates the agent display of the 
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selected personae using the utterances and effect selected. 
Data is referenced and updated in the student data object by 
this processing, in particular fields reflecting the 
student's agent personalization choices, such as the desired 
5 personae, and fields reflecting recent agent behaviors • 

Also illustrated is communication materials 615. These 
materials manage and provide resources for various group 
activities, such as student tutoring, group work with 
particular materials, and group contests. As indicated by 
10 arrow 614, these materials access data subtypes preferably 
stored in school-student data 615. 

5.4»2* The Bxacutive Software 

The executive software (••ES") collects together a number 

15 of components which customize the operating system to the 
requirements of this invention and, if needed, also extend 
it. ES software implements common and special facilities. 
Exemplary common facilities include task control, 
communications, and I/O facilities; exemplary special 

20 facilities include a preferred animation facility and a 
client logon and startup facility. In a preferred 
implementation, ES software is built as frontends, wrappers, 
or extensions to available OS software components. For 
example, certain facilities can be implemented in C or C-hh- 

25 and directly call OS interfaces, such as the Windows™* 
graphics device interface or its extensions. Other 
facilities can be built as classes using available 
implementation language libraries, such as the packages in 
Java™ 1.0. Yet other facilities can be provided directly as 

30 part of software packages, such as the display I/O functions 
present in commercially available authoring packages. 

The task control facility manages the startup of system 
components. First, it verifies that the student is permitted 
to activate a requested component by checking schedules and 

35 priorities with the schedule /calendar tool. Indirectly, the 
student's agent can be queried to determine if this 
activation is reasonable. Next, if permitted, this facility 

- 76 - 



— BNSOOCID: <WO. .9744766A1 _L> 



wo 97/44766 




PCT/US97/08685 



manages the loading of required software from server systems 
and the accessing of data objects. Finally, task control 
then starts up the component by making any necessary OS 
calls. Task control also notifies other system components at 
5 task terminations. 

The communications facility manages network 
communications and provides whatever client-server or file 
server protocols are not present in the OS. Such protocols 
might include HTTP V2 • 0 with URL, name resolution. It 

10 maintains all necessary network sessions, including sessions 
with the major classes of servers, including executable 
software servers and instructional material servers. In the 
case of portable student data objects, communication is also 
maintained with the registry function. In case of downloaded 

15 student data objects, communication is maintained with 

student data objects repositories. Data snip libraries can 
reside on any of these server classes. In alternative 
implementations these servers can be on one or several 
physical server systems. Finally, this facility handles all 

2 0 remote access requests for information including requests for 
downloading and uploading. 

The I/O facilities includes input and output display 
handlers for object level display I/O, The display handlers 
receive object level requests for text, graphics, video and 

2 5 audio and translate them into whatever interface is supported 

by the OS. The input handler receives low-level inputs from 
the OS input handlers and processes them according to the 
current screen format. In certain cases, it can directly 
call the output handler to perform immediate feedback 

3 0 actions, such as highlighting or dragging an object. In 

other cases, it can pass I/O event messages in an appropriate 
format to the system component owning that screen object, for 
example, the on-screen agent. The output handler receives 
object presentation specifications, such as system owner, 
35 position, characteristics, size, any animation information, 
and whether the object is an input focus, then updates any 
necessary screen mapping, and generates OS requests to 
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display the object. An example of such an object 
specification includes a selectable text field object with 
specified contents, perhaps scrollable, displayed by a 
particular instructional material, 
5 Exemplary specialized ES facilities are animation and 

client startup. it is preferable that the client system 
support animation, which is a connected and timed sequence of 
displays potentially calling on all display modalities 
available, and other timed presentations. Although this 

10 invention is adaptable to any suitable animation facility, a 
preferred facility presents a script based interface. 

The inputs to an animation facility are scripts which 
comprise object display commands/ the timing of the object 
displays, an input specification of how any permitted user 

15 inputs are to be handled, and the types of events to be 
returned to the initiating system component. Examples of 
display events include user input causing script branching 
and termination or interruption. The animation facility 
receives a script from another system component, requests 

20 downloading of necessary display objects, for example from 
data snip libraries, interprets the timing specification, and 
sends request to the input and output handlers. 

The ES startup facility is described herein with respect 
to a student client to run on a network attached computer. 

25 Startup is similar for other client systems, including 

teacher and instructional designer clients. Each component 
of the ABI system must be downloaded from a server when 
needed or accessed from portable media, if as is typical, no 
component of the system is resident on a client prior to 

30 startup. The initial step involves the student accessing any 
client attached to an ABI network. Accessing can involve as 
little as powering-on the computer and requesting startup of 
the system on this client. Preferably, accessing by a 
student user begins by accessing the server with the system 

35 manager, for example, using a standard intranet browser which 
can be resident on the client or downloaded by a power-on 
bootstrap process. The student logs on to the system 
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manager, which then performs authentication, for example, by 
means such as password or identification card. Upon 
successful authentication, the system manager downloads and 
starts the ES. The ES then initiates necessary communication 
5 sessions, including those with the system servers, and then 
downloads the session manager software, the agent software, 
and the scheduler/ calendar tool software. Preferably, the 
portable student data object in conjunction with the ES then 
authenticates the student and the student data object data is 

10 accessed from the portable media. Alternatively, the student 
data object is downloaded from a server. Then the session 
manager presents the student display in a form depending on 
student preferences in the student data object and receives 
input from the system area of the display. The 

15 schedule/calendar in cooperation with the agent then 

determines what materials the student is to be presented with 
or can select from. Finally, the materials data and engine 
are then downloaded and the substance of the student session 
commences . 

20 Presentations of personae or merely appropriate coherent 

responses can be created from audio and video display- objects 
downloaded from a server and referenced upon demand. These 
display objects, or data snips, can be linked into groups 
corresponding to the particular presentations. Further, 

25 these linked groups can include small pieces of code that 
allow for a branching and interactive short meta-response . 
Thus, one meta-response can include opportunities for student 
input with the next display object being chosen in response 
to that input. Each of these groups is catalogued and 

30 referenced as that persona behaving in a particular manner. 

5.4«3. ^Y^^mm Event Procaaaina 

The ABI System on the student client, and on other 
clients, is advantageously organized to wait for student and 
35 timer input events and then to respond appropriately to these 
events. This is common current practice for constructing 
interactive systems. However, this invention is adaptable to 
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practices that might be developed in the future for 
constructing interactive systems as long as the principal 
system components and their mutual interactions as previously 
described can be represented in such a future practice. 
5 With reference to Fig. 7, this section sequentially 

describes the system processing of an input event and its 
consequences from initial input until the system waits for 
the next event » This description provides detail of how the 
system components communicate with each other and of how 

ao information and control flows through the system. More 

detailed description of the individual components appears in 
following sections. 

Fig. 7 conceptually illustrates control flow and event 
processing in a student client. Generally, at the left of 

15 this figure is relevant data; in the center is processing 

steps; and at the right are I/O events- As is apparent from 
the context, the arrows in this diagram represent either data 
references or control flow. The control flow is at times 
implemented by information bearing messages and at other 

20 times effected by standard ES and OS system calls. Further, 
this figure is directed to the processing of those components 
that interact with the student's agent. Those components, 
such as the session manager, which do not interact with the 
agent can be constructed according to standard interactive 

25 techniques (see, e.g., Petzold, 1996, ProgT -aTtip ina windQwg<£> 
95, Microsoft Press, Redmond WA) . 

Thus, at wait point 701 the system waits for the next 
student input or timer event. This wait commences at wait 
points 707, 713 and 717 after the system completes processing 

30 of the previous iavent, or possibly after system startup. 
Upon receiving an input event, ES I/O handlers 702 decide 
whether the event represents an action without significance 
for other system components. For example, the event can be a 
student action causing only the highlighting of an object or 

35 a timer event directing an animation. Such events can be 
entirely processed by the I/O handlers, and system waiting 
resumes at 701. If the event represents a student' request 
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for the materials engine 703 or a meta-request for the agent 
711, the I/O handlers format it appropriately and 
communicates it to the correct component owning that input 
object. If the event represents a time interval set by one 
5 of the materials engines or by the agent, it is formatted and 
passed to the requesting component. in the case of group 
work, the event can be generated on a remote computer system 
and transmitted over a network link in order to reach the 
local I/O handlers. 

10 The I/O handlers format event messages into the format 

expected by the component they are directed to. They include 
in each message an indication of the type of student action 
or time interval along with any input information from the 
action. One input event can generate several events 

15 messages. 

Herein, messages directed to materials engines 703 are 
considered first; messages directed to agent action 
processing 711 are considered second. Although Fig. 7 
illustrates two materials engines, one engine or more than 

20 two engines are possible on a system from time to time. 

Regardless of the number of materials engines present, event 
messages are communicated to the correct engine. The 
materials engines 703 process a plurality of read-only 
n^aterials data 704 representing instructional materials, 

25 tools, and communication materials. As indicated by arrow 
725, these engines also access control information determined 
by agent action processing 7ii. This control information can 
be accessed in any convenient manner, such as by an exchange 
of messages or by referencing parameters stored in a shared 

30 memory region* This information controls the materials 
engines at educationally significant points during their 
materials presentation* Optionally, materials engines 703 
can also access remote databases 7 05 and other remote 
resources available through the system network. The engines 

35 use these two sources, and optionally three sources, of input 
to cause presentation 706 to the student of, for example, an 
instructional lesson, a homework exercise, an instructional 
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tool, or a joint educational contest. This presentation uses 
the I/O handler of the ES and OS to generate actual student 
display. 

As indicated by arrow 727, the materials engines also 
5 generate messages directed to agent action processing 71 i. 
At educationally significant points during materials 
presentation, notations in the materials cause the engines to 
format a message to the agent. These messages also include 
an indication of event type and relevant data, perhaps 

10 including timing data. One student input can generate 
several agent messages. At points which are not 
educationally significant, the materials can not generate 
agent messages. Thus, the system can wait again for the next 
student action at wait point 707 if the previous student 

15 action had either no input significance for the materials or 
agent or no educational significance for the agent. 

However, if messages have been sent to the agent, agent 
action processing 711 is activated. In the preferred but not 
limiting embodiment, this processing is table based. Rules 

20 from policy filter table 708 are evaluated in view of the 
data included in an incoming event message and in view of 
data on the student's past performance and the student's 
pedagogic model in student data object 712. For example, 
comparisons with student history are needed to determine 

25 relative performance. These rules propose candidate agent 
actions. Next, rules in decision weight table 709 and in 
selection criteria table 710 filter the proposed actions into 
a final list of agent actions. These final actions can cause 
the update of the information in student data object 712 with 

30 data on the student's current performance and behavior, as 
indicated by double headed arrow 728. They can also make 
available control information for the materials, as indicated 
by arrow 725. Finally, and importantly, the actions can 
include display actions for causing visible agent behavior. 

35 The list of final display actions is sent to agent behavior 
processing 716. It is possible that no visible agent 
behavior is appropriate and that the list of display actions 
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is empty- In this case the system waits for further student 
input actions at wait point 713. 

Agent behavior processing 716 processes the final list 
of display actions communicated from agent action processing 
5 711 in view of utterance tables 714, display behavior tables 
715, preferences and the record of recent agent behavior in 
student data object 712 in order to generate a coherent 
display of the on-screen agent personae reacting to the 
previous actions. The final actions select utterance 

10 templates which are completed with parameters contained in 
the actions. The personae from the display behavior table 
selected according to the student preferences use these 
complete utterances to generate display objects, and perhaps 
animation scripts or applets, which are sent to the I/O 

15 facilities of the ES for final display 718. Agent behavior 
processing also updates the student data object with 
information concerning this current behavior* 

Having completed all possible processing of the student 
input action, the system now waits at wait point 717 for the 

2 0 next student action or time interval. 

5.5, The Instructional Materials And The Tools 

This section describes a preferred embodiment for the 
instructional materials, the student tools, and 

2 5 communications materials. In this embodiment, the materials 
have a uniform structure, being defined by materials data 
which is used by the materials engine to appropriately 
generate displays and perform functions. This uniform 
structure permits a uniform handling of the interface between 

30 all the materials and the agent. In alternative embodiments, 
certain tools and the group communication materials, for 
example, can be advantageously separately implemented as 
separate progrsuas that themselves maintain the necessary 
agent interface. Such certain tools include a calculator, a 

35 dictionary, an encyclopedia, and group communications. in 
further alternative embodiments, each instructional material 
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could be a separate program that also maintained the 
necessary agent interface. 

This section first describes the general structure of 
the instructional materials and then describes the tools 
5 typically available on an ABI system. 

5,5. 1. Thm Instrttctionml MatTialg 

In a preferred embodiment a common materials engine 
interprets specific materials data to perform instructional 

10 and tools functions. These are described with reference to 
instructional materials with adaptations needed for the other 
materials noted- Turning first to the materials data, in the 
preferred embodiment it includes three principal sections for 
presentation items, sequencing logic, and notations. The 

15 presentation items include whatever is displayed, preferably 
represented as display objects, which can be parametrized. 
These display objects can be packaged with the materials data 
or can be downloaded from a server on demand. The notations 
contain additional data related to the materials display. 

20 These include, for example, prerequisites, links to related 
inaterial, expected student performance, and help and hints - 
The notations are preferably generated from templates 
referencing parameters from the materials data and student 
performance inputs. The materials engine uses notations to 

25 generate messages to the agent, which comprise one part of 
the agent interface. The sequencing logic is executable or 
interpreted code that animates the particular materials. It 
references all data in the particular materials to cause the 
ordered display of the presentation items and to send 

30 messages to the agent according to the notations* 

The materials data is advantageously grouped into 
entries, each entry representing a minimum item of 
presentation, which can, however, involve several screens. 
These entries are preferably specialized at least into a 

35 header or initialization entry and the other entries. Table 
2A illustrates a typical materials header entry which is the 
first entry processed when the materials are initialized. 



wo 97/44766 ^ 




PCT/US97/08685 



TABLE 2 A: HEADER ENTRY 



5 



Instructional 
Header Entry- 






Presentation 
items 


Sequencing Logic 


Notations 




Global variables; 
Global functions; 
Local variables; 
Local functions; 


Prerequisites for 
materials ; 
Reference to other 
materials or standard 
texts. 

Agent initializing 
information including : 

- meaning of global 
variables ; 

- education paradigm 
adopted 

available helps, tools 



15 

This header has several particular features. The sequencing 
logic for the header frame consists largely of definitions of 
variables and functions. The global variables are shared 
between system components and include the control variables 

2Q that the agent sets and that the sequencing logic references 
and tests in order to be guided by agent control. These 
variables, which comprise another part of the agent 
interface, for example, can control the time pacing of 
instruction, the new concept seeding rate, the density of new 

25 examples, the time pacing, or the difficulty of 

discriminations. Local variables are available to the 
sequencing logic during materials processing- Global 
functions are those global system functions that can be 
called by the sequencing logic. Also, dictionary lookup, 
spell checking, or encyclopedia lookup can be globally 
implemented and shared. In a Windows"" environment, the 
global functions can be DLLs- Finally, local functions are 
available locally to the sequencing logic. For example, in a 
scientific calculator tool, computations can be local 

35 functions. For a further example, in instructional materials 
the local functions are available for checking user inputs 
for correctness, scoring quizzes, and so forth. 
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The notations in the header entry generate materials 
initialization messages to the agent. These messages inform 
the agent about these materials, about what global variables 
they respond to, about what helps, hints, and tools are 
5 useful, and importantly about the educational paradigm the 
materials use. Preferably, information about this paradigm 
is structured according to the instructional materials 
interface standard. See infra. The notations can also 
contain additional information, such as prerequisites for the 
10 whole materials and references to other materials and texts. 
Finally, the presentation item can be, for example, an 
introductory screen. 

Table 2B illustrates a typical entry which is processed 
during regular materials presentation. 

15 

TABLE 2B: KATERIAL8 ENTRY 



20 



30 



Instructional 
Entry N 






Presentation 
items 


Sequencing Logic 


Notations 


Display 
objects , 
possibly 
including 
those for 
text, 
graphics, 
sound, 

animation, or 
video 


Interpreted or 
executed control 
statements 
referencing 
variables , 
notations, and 
any student 
input, and 
causing display 
of display 
objects and 
execution of 
notations 


Prerequisites for frame; 
Expected timing, and 
difficulty ; 

Relation to other frames; 
Links to other materials 
and sources. 
Changes to agent 
initializing information ; 
Student performance data ; 
Student error data 



For this entry, the presentation items are those for the 
materials display. The sequencing logic causes this display 
in view of the variables and other information in the 
materials data and any student input. Finally, the notations 
result in agent messages reporting changes in any parameters 
set at initialization, student performance data, student 
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errors, and other educationally significant information. The 
notations can also contain information specific to this 
frame, such as expected difficulty and timing. 

Turning now to the materials engine, its first 
5 processing step is to request the executive software to 

download the requested materials data from the instructional 
materials server. It next processes the header entry, links 
to global variables and functions, and sends initialization 
event messages to the agent. vmen presentation begins, it 

10 interprets or calls for execution the sequencing logic on the 
first frame. From this frame it proceeds to activate other 
frames as directed by the sequencing logic. Lastly, when the 
materials processing ends, any termination messages as 
directed by the notations are sent to the agent and the 

15 materials are deleted from the client. 

In a preferred embodiment, the materials are all 
implemented similarly. Most differences between the 
instructional materials, tools, and communication materials 
are in the presentation items and the sequencing logic, 

20 including different global and local entities. All materials 
are expected to have notations for generating agent messages 
that record materials initiation and termination and student 
performance and errors. Preferably this information is 
reported in a standardized manner according to an 

25 instructional materials interface standard. See Infra, 

5.5.2. The Tools 

The ABI system is adaptable to a wide range of necessary 
and optional student and teacher tools tailored to the 

3 0 students and the courses of instructions. In the case of 

elementary education, the following preferable tools include 
certain general tools and the communication, or joint work, 
materials. In the case of commercial education, other tools 
can be preferable. The schedule/calendar tool participates 

35 in permitting access to materials according to student 
schedule and is preferably found in all embodiments. 
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The discussion in the section is directed to an 
implementation for elementary education. It is not limiting 
in the tools that can be used in an implementation. 

5 5. 5. 2,1. Th# GenTal Toola 

The general tools are preferably present in a range of 
forms selected according to data in the school-student data 
areas, including grade level, and teacher permissions. one 
general tool is a calculator, which can have forms varying 

10 from a simple four function calculator to a complex graphing 
calculator. Other general tools include lancfuage tools, such 
as a spelling checker, a thesaurus, a word pronouncer, an 
encyclopedia, and a dictionary. Different levels of each 
language tool can be provided suitable for different grade 

15 levels. In an alternative embodiment, the language tools can 
be integrated with a "starfish" tool, which allows the user 
to place the center of a starfish" over a particular word, 
and then, by selecting the appropriate arm of the starfish to 
obtain the definition, a pronunciation of the word, a rhyming 

2 0 word, a synonym, or an antonym. Another general tool is a 

word processor, perhaps with a drawing mode which can be 
provided as a multi-level set of writing and drawing tools. 
The writing and drawing capabilities available to a student 
are selectable by the student or teacher, A last general 
25 tool is a link -maker, which offers exercises in various types 
of memorization, such as paired associates, serial learning, 
ordered serial learning, and mnemonics. Finally, additional 
tools can be added to an implementation to meet specific 
educational needs. For example for geography lessons a map 

3 0 tool can be added. For student projects, an encyclopedia 

tool and a network search tool can be added. The Study 
Buddies'" can provide instruction in operation of the tools 
such as the use of keywords and operators. Specialized tools 
can be added for commercial or industrial training. 
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